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This  TR  contains  a  dissertation  submitted  in  partial  fulfillment  of  the  requirements  for 
the  degree  Doctor  of  Philosophy.  I  discuss  here  a  general  model  for  compilers  that 
take  imperative  code  written  for  sequential  machines  {ordinary  code)  and  detect  the 
parallelism  in  that  code  that  is  compatible  with  the  semantics  of  the  underlying  pro¬ 
gramming  language.  This  model  is  based  on  the  idea  of  separating  the  concerns  of 
parallelism  detection  and  parallelism  exploitation. 

This  separation  is  made  possible  by  having  the  detection  component  provide  an 
explicit  representation  of  the  parallelism  available  in  the  original  code.  This  expli¬ 
citly  parallel  representation  is  based  on  a  formalization  of  the  notion  of  permissible 
execution  sequences  for  a  given  mass  of  code.  The  model  discussed  here  prescribes 
the  structure  of  the  parallelism  detector.  This  structure  depends  (1)  on  recognizing  a 
hierarchical  structure  on  a  graph  representation  of  the  program,  and  (2)  on  separately 
encoding  parallelization  conditions  and  effects. 

Opportunities  for  parallelization  can  then  be  discovered  by  traversing  the 
hierarchical  structure  from  the  bottom  up.  During  this  traversal,  progressively  larger 
parts  of  the  program  are  compared  against  the  independently  encoded  conditions,  and 
transformed  when  the  conditions  are  satisfied.  The  hierarchy  guarantees  that  the 
results  of  transforming  a  piece  of  a  program  propagate  in  time  to  affect  the  possible 
parallelization  of  larger  pieces.  Although  some  of  the  algorithms  used  have  exponen¬ 
tial  worst  cases  for  general  graphs,  their  observed  behavior  on  real  flow  graphs  is  no 
worse  than  quadratic  on  the  size  of  the  original  program. 
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code  that  is  compatible  with  the  semantics  of  the  underlying  progranuning 
language.  This  model  is  based  on  the  idea  of  separating  the  concerns  of  parallelism 
detection  and  parallelism  exploitation. 

This  separation  is  made  possible  by  having  the  detection  component  provide  an 
explicit  representation  of  the  parallelism  available  in  the  original  code.  This 
explicitly  parallel  representation  is  based  on  a  formalization  of  the  notion  of 
permissible  execution  sequences  for  a  given  mass  of  code.  The  model  discussed  here 
prescribes  the  structure  of  the  parallelism  detector.  This  structure  depends  on  (1) 
recognizing  a  hierarchical  structure  on  a  graph  representation  of  the  program,  and 
(2)  separately  encoding  parallelization  conditions  and  effects. 
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hierarchical  structure  from  the  bottom  up.  During  this  traversal,  progressively 
larger  parts  of  the  program  are  compared  against  the  independently  encoded 
conditions,  and  transformed  when  the  conditions  are  satisfied.  The  hierarchy 
guarantees  that  the  results  of  transforming  a  piece  of  a  program  propagate  in  time  to 
affect  the  possible  parallelization  of  larger  pieces.  Although  some  of  the  algorithms 
used  have  exponential  worst  cases  for  general  graphs,  their  observed  behavior  on 
real  flow  graphs  is  no  worse  than  quadratic  on  the  size  of  the  original  program. 
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Abstract 


This  dissertation  discusses  a  general  model  for  compilers  that  take  imperative  code 
written  for  sequential  machines  {ordinary  code)  and  detect  in  that  code  the 
parallelism  that  is  compatible  with  the  semantics  of  the  underlying  programming 
language.  This  model  is  based  on  the  idea  of  separating  the  concerns  of  parallelism 
detection  and  parallelism  exploitation. 

This  separation  is  made  possible  by  having  the  detection  component  provide  an 
explicit  representation  of  the  parallelism  available  in  the  original  code.  This 
explicitly  parallel  representation  is  based  on  a  mathematical  formalization  of  the 
notion  of  permissible  execution  sequences  for  a  given  mass  of  code. 

Having  made  that  separation,  one  can  discuss  an  organization  for  the  parallelism 
detection  component.  This  organization  depends  (1)  on  recognizing  a  hierarchical 
structure  on  a  graph  representation  of  the  program,  and  (2)  on  separately  encoding 
parallelization  conditions  and  effects. 

Opportunities  for  parallelization  can  then  be  discovered  by  traversing  the 
hierarchical  structure  from  the  bottom  up.  During  this  traversal,  progressively  larger 
parts  of  the  program  are  compared  against  the  independently  encoded  conditions,  and 
transformed  when  the  conditions  for  a  parallelizing  transformation  are  satisfied. 
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CHAPTER  1 


Introduction 


1.1.  The  Problem 

Compilers  for  sequential  imperative  code  are  understood  in  terms  of  a  well-accepted 
model:  a  lexical  analyzer  tokenizes  the  input  text,  a  syntactic  analyzer  parses  the  tok- 
enized  input,  a  semandc  analyzer  validates  non-context-free  attributes  of  the  parsed 
input,  etc.  (For  examples,  see  [ASU86,  fig.  1.10]  and  [WaG83,  §1.4].)  Even  when  a 
compiler’s  code  does  not  clearly  exhibit  this  division  of  labor,  this  model  helps  to 
understand  it,  or  to  compare  it  with  similar  compilers. 

Thus  the  standard  model  is  useful  in  two  senses.  In  the  descriptive  sense,  it 
describes  the  behavior  of  existing  compilers.  In  the  prescriptive  sense,  it  lays  out  the 
sti'ucture  of  compilers  yet  to  be  written,  thereby  giving  their  builders  a  mental  map  of 
what  needs  to  be  done. 

In  this  context,  let  a  parallelizer  be  a  compiler  that  converts  ordinary  code  for 
execution  on  parallel  machines.  By  ordinary  code  I  mean  code  written  in  a  sequen¬ 
tial  imperative  language,  without  concern  for  parallelism.  There  already  exist  many 
successful  parallelizers.  However,  they  have  all  been  conceived  and  implemented  as 
special  cases,  carefully  tuned  to  specific  combinations  of  input  language  and  target 
architecture.  Although  they  are  impressive  .ccomplishments  of  the  art  of  compiler 
design,  there  is  but  the  vaguest  impression  of  a  generalizable  design  in  them.  More¬ 
over,  there  isn’t  a  consensus  about  a  model  for  them,  at  least  not  to  the  successful 
extent  of  the  standard  model  for  sequential  compilers. 

The  goal  of  this  dissertation  is  to  provide  a  model  for  paralfelizing  compilers, 
and  to  show  that  the  model’s  descriptive  and  prescriptive  powers  are  similar  to  those 
of  the  standard  model. 

1.2.  A  Model 

The  model  I  present  here  is  based  on  the  separation  of  two  concerns:  detection  of 
parallelism  and  exploitation  of  parallelism.  When  designing  a  new  parallelizer  (or 
trying  to  understand  an  existing  one),  my  model  recommends  isolating  the  task  of 
discovering  what  parts  of  the  program  can  be  done  in  parallel  (regardless  of  whether 
doing  so  buys  us  something)  from  the  task  of  selecting  a  particular  parallel  execution. 
The  first  task  depends  only  on  the  semantics  of  the  given  programming  language;  the 
second  depends  mainly  on  the  details  of  the  machine  that  will  run  the  object  code.^ 


'  This  separation  has  consequences  elsewhere  in  parallel  programming.  For  instance,  Crowl  [Cro91] 
argues  for  its  importance  in  achieving  architectural  adapuibility  of  (explicitly)  parallel  programs. 
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Figure  .  1  represents  this  idea.  The  Analyzer  is  in  charge  of  detecdng  parallel¬ 
ism,  while  the  Synthesizer  is  in  charge  of  constructing  a  profitable  parallel  version  of 
the  originally  sequential  program. 

The  communication  betv/een  these  two  components  is  a  critical  a.spect  in  this 
work.  I  will  discuss  later  why  the  separation  of  concerns  requires  that  the  flow  from 
the  analyzer  to  the  synthesizer  should  consist  of  an  Explicitly  Parallel  Representation 
of  the  program. 

This  dissertation  is  concerned  mainly  with  the  Analyzer  (the  parallelism  detec¬ 
tion  component).  In  this  model,  the  source  program  is  first  validated  and  transformed 
by  a  classic  front  end.  The  output  of  that  first  part  of  the  compiler  is  a  graph 
representation  of  the  program  (which  I  will  call  the  program  graph ).  I  will  assume 
here  that  the  source  program  gets  reduced  to  a  control  flow  graph  at  some  granularity 
(statements  or  basic  blocks  are  examples).  This  is  the  usual  representation,  see  for 
instance  [ASU86,  §9.4  and  §10.1].  I  will  call  each  node  of  that  graph  a  granule,  a 
generic  term  that  avoids  pre-judging  the  actual  resolution  of  the  analysis. 
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In  addition,  I  will  assume  that  data  flow  information  at  the  granule  level  is  either 
available,  or  conveniently  computable  from  the  output  of  the  front  end.  The  front  end 
computes  this  information  (control  and  data  flow)  on  the  assumption  that  the  program 
will  be  executed  seq-  entially.  This  assumption  defines  a  partial  order  on  the 
sequence  of  execution  of  the  various  parts  of  the  program.  The  purpose  of  the  paral¬ 
lelism  detector  can  be  described  as  finding  a  relaxation  of  this  partial  order. 

The  first  stage  of  the  analyzer  proper  is  the  Fragment  Enumerator .  This  com¬ 
ponent  will  select  pieces  of  the  program  (ITl  call  them  gtntncdWy  fragments)  to  be 
considered  for  parallelization.  Fragments  formalize  a  bottom-up  strategy  for  the 
analysis.  Therefore,  I  need  to  define  fragments  in  such  a  way  that  some  containment 
relation  holds  between  them.  Some  basic  properties  are  shared  by  all  useful 
definitions  of  fragments: 

(1)  Some  minimal  subparts  of  the  program  (perhaps  individual  granules)  are 
fragments 

(2)  The  whole  program  is  a  fragment,  and 

(3)  Some  intermediately  sized  configurations  are  fragments  too. 

Every  choice  of  fragment  definition  produces  a  hierarchy  of  parts  of  a  program,  of 
progressively  increasing  size. 

However,  not  all  hierarchies  are  equally  useful.  It  is  possible  to  construct 
hierarchies  whose  components  don’t  abstract  useful  properties  of  the  program,  or  that 
abstract  at  a  grain  too  fine  or  too  coarse  for  practical  application.  Choosing  a  suitable 
definition  of  fragment  (and  hence,  of  fragment  hierarchy)  will  thus  be  critical  to  the 
success  of  a  parallelism  detector. 

If  we  further  restrict  our  attention  to  the  definitions  that  yield  practical  hierar¬ 
chies,  the  choice  between  granularities  permits  us  to  control  the  tradeoff  between 
thoroughness  and  cost  of  the  analysis. 

Parallelization  in  this  scheme  is  driven  by  the  fragment  hierarchy.  Fragments 
are  presented  in  turn  to  the  transformer ,  an  agent  in  charge  of  exposing  the  parallel¬ 
ism  in  a  fragment.  The  discovery  of  this  parallelism  occurs  by  subjecting  the  frag¬ 
ment  to  a  series  of  tests,  which  involve  conditions  on  the  shape  of  the  fragment,  its 
position  in  the  fragment  hierarchy,  its  associated  data  flow  information,  and  the 
results  of  previous  parallelizations. 

When  a  fragment  is  being  considered,  I  would  like  to  take  advantage  of  the 
results  of  having  previously  considered  its  sub-fragments.  So,  the  fragment  enumera¬ 
tor  must  guarantee  the  following  condition:  If  f  is  a  fragment  of  a  program  graph, 
then  f  is  eventually  presented  to  the  transformer.  If  f  and  g  are  fragments  of  a 


^  I  assume  here  that  fragment-level  flow  information  can  be  computed  from  the  basic-block-level  flow 
iiiformation  provided  by  the  front  end.  (This  is  another  constraint  on  the  definition  of  a  fragment.)  As 
fragments  contain  blocks,  well-known  technology  to  propagate  flow'  information  bottom-up  applies 
here. 
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program  graph,  and  f  is  a  proper  subgraph  of  g,  then  g  is  last  presented  to  the 
transformer  after  the  last  presentation  of  f.  This  condition  guarantees  a  form  of  ord¬ 
erly  progress  in  the  enumeration,  as  it  roughly  states  that  large  eventually  follows 
small.  I  will  call  this  condition  eventual  enumeration  ordering .  (Notice  that  a  frag¬ 
ment  can  be  presented  for  parallelization  more  than  once.) 

The  parallelism  (or  its  lack)  exposed  by  the  transformer  is  encoded  in  an  execu¬ 
tion  set.  An  execution  set  for  a  fragment  is  a  set  of  permutations  of  parts  of  the  frag¬ 
ment,  tfiat  contains  only  permutations  whose  sequential  execution  honors  the  data  and 
control  dependences  in  that  fragment.  Exposing  parallelism  in  the  fragment  reduces 
to  finding  larger  execution  sets  (Chapter  3). 

In  order  to  expose  these  larger  execution  sets,  the  transfonner  may  need  to 
modify  the  fragment  (I  call  this  a  reorganization )  or  add  some  code  to  the  rest  of  the 
program  (and  this  I  call  a  compensation-,  these  two  terms  are  standard).  The 
transformer  does  not  have  hard-coded  information  about  the  needed  tests  and 
transformations:  the  required  tests,  reorganizations,  compensations,  and  calculations 
of  execution  sets  are  encoded  in  another  structure,  the  parallelizations  catalog . 

Eventually,  the  enumerator  will  present  for  possible  transformation  the  fragment 
corresponding  to  the  total  program,  for  the  last  time.  After  that  consideration,  the 
resulting  execution  set  is  the  final  output  of  the  analyzer. 

Figure  1.2  shows  the  flow  of  information  within  the  Analyzer.  To  a  first  approx¬ 
imation,  the  Analyzer  is  a  pipeline  con.posed  of  three  stages.  The  pipeline  model  is 
not  entirely  adequate,  because  each  stage  keeps  state  that  is  visible  to  the  others,  but 
will  serve  us  for  an  introduction.  So,  let’s  follow  the  left-to-right  pointing  arrows  in 
Figure  1.2. 

The  first  stage  is  the  Fragment  Enumerator.  This  consumes  (notionally)  the  pro¬ 
gram  graph  and  produces  fragments.  Those  fragments  are  given  execution  sets  by  the 
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Figure  1.2.  Information  Flow  in  the  Analyzer 
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second  stage,  the  Transformer.  Finally,  the  last  stage  in  the  figure,  the  E.P.R.  genera¬ 
tor,  outputs  the  final  execution  sets,  in  a  notation  useful  to  the  synthesizer  com¬ 
ponent.  That  notation  is  what  I  call  an  Explicitly  Parallel  Representation .  (Of 
course,  the  E.P.R.  can  just  coincide  with  whatever  notation  we  use  for  execution 
sets.) 

The  stages,  as  said  above,  keep  state  too.  The  solid  double-headed  arrows 
represent  creation  and  modification  of  two  major  data  structures:  the  Fragment  and 
ExecSet  hierarchies.  These  are  effectively  shared  by  all  the  stages.  In  addition,  a 
static  structure  is  queried  by  the  Transformer:  the  Parallelizations  Catalog,  which 
encodes  the  transformations  that  have  been  deemed  worth  trying. 

Finally,  the  dashed  double-headed  arrows  represent  a  more  serious  potential 
departure  from  the  pipeline  model.  The  transformer  could  (by  virtue  of  reorganiza¬ 
tion  oi  ^.ompensation)  modify  the  flow  structure  of  the  program  to  the  extent  of  forc¬ 
ing  the  calculation  of  a  new  Fragment  Hierarchy.  As  the  possible  new  fragments 
would  require  execution  sets  too,  the  ExecSet  hierarchy  would  be  naturally  modified 
too. 

1.3.  The  Thesis 

The  thesis  defended  by  this  dissertation  is  composed  of  two  claims. 

The  first  claim  is  that  the  detection  of  parallelism  in  ordinary  programs  can  be 
perfomied  independently  of  the  architecture  of  the  machine  that  will  un  them, 
thereby  separating  this  detection  from  the  problem  of  selecting  useful  parallelism, 
and  that  such  separation  leads  to  a  very  general  model  of  how  parallelizers  work. 

I  will  support  this  claim  by  showing  a  model  of  parallelization  that  realizes  this 
separation. 

The  second  claim  is  that  the  generality  obtained  by  this  model  is  not  a  serious 
impediment  to  effective  implementation.  It  is  at  least  conceivable  that  the  desired 
separation  would  sever  important  links  between  the  two  parts  posited,  and  that  recov¬ 
ering  that  connection  could  be  costly.  Or,  it  could  be  that  the  generality  of  the  model 
depends  on  algorithms  that  are  inherently  too  expensive.  In  either  case,  compilers 
designed  around  my  model  could  well  be  too  slow'  to  compete  practically  with  others 
crafted  in  less  systematic,  but  adequate,  ways.  I  claim  that  no  such  crippling 
inefficiencies  arise  from  my  model. 

I  will  support  this  claim  by  observing  the  behavior  of  a  proof-of-concept  imple- 
mentation  of  a  parallelism  detector  for  a  subset  of  C  [KeR78],  named  SPOIL. 

About  the  subset  of  C  compiled  by  SPOIL,  suffice  it  to  say  that  it  is  a  simple 
imperative  language,  with  only  first  order  features  (various  scalar  types,  some  struc¬ 
tured  types  like  records  and  arrays,  functions,  some  fomi  of  looping).  The  target 


^  Systematic  Parallelization  Of  Imperative  Languages.  Note  also  the  play  on  the  fact  that  to  compile 
derives  from  a  French  word  that  used  to  mean  to  plunder. 
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machine  is  left  unspecified,  as  it  is  part  of  the  claim  that  this  is  not  a  matter  with 
which  the  parallelism  detector  should  concern  itself. 

The  rest  of  this  dissertation  discusses  the  parallelism  detection  component  of  a 
parallelizer  for  a  generic  imperative  language,  and  describes  SPOIL  as  an  application 
of  this  model  for  the  specific  case  of  C. 

It  is  important  to  pay  attention  to  three  levels  of  abstraction  represented  in 
SPOIL.  At  the  highest  level,  SPOIL  is  a  representative  of  the  compilers  that  follow  my 
model.  Thus,  it  separates  detection  from  exploitation,  is  driven  by  a  hierarchy  of  pro¬ 
gram  parts,  separates  the  compiler’s  logic  from  the  specific  transformations  applied, 
and  represents  parallelism  explicitly.  At  this  level,  SPOIL  can  be  judged  by  whether 
(or  how  well)  it  adheres  to  the  model.  Conversely,  SPOIL  helps  us  judge  the  useful¬ 
ness  of  the  basic  tenets  of  the  model,  by  providing  existence  proof  of  the  model’s 
applicability. 

Among  those  compilers  that  follow  the  model,  SPOIL  is  distinguished  (second 
level)  by  specific  definitions  of  granule,  fragment,  and  aspect  (these  will  be  intro¬ 
duced  later)  and  by  a  specific  catalog  of  parallelizing  transformations.  At  this  level, 
SPOIL  can  be  judged  by  how  well  it  succeeds  in  using  those  concepts  to  parallelize  the 
specific  subset  of  C  that  it  compiles.  Conversely,  having  observed  its  success  in 
those  terms,  SPOIL  helps  establish  the  claim  that  languages  in  such  class  can  be 
treated  by  my  model.  The  critical  feature  of  C  used  at  this  level  is  its  amenability  to 
effective  data  and  control  flow  analysis.  That  the  class  of  languages  for  which  this 
level  applies  is  well-defined  can  be  seen  by  considering  that  there  are  imperative 
languages  that  don’t  fit  this  characterization  (say,  the  higher-order  languages  studied 
in  [Ney88]). 

Finally,  at  the  third  (and  lowest)  level,  SPOIL  is  a  particular  Common  Lisp  pro¬ 
gram  running  under  the  UNIX  operating  system  and  compiling  specific  C  codes.  At 
this  level  one  could  judge  the  implementation  characteristics  of  SPOIL,  like  its 
efficiency  and  effectiveness  on  a  test  suite.  That  these  parameters  end  up  being 
acceptable  (see  Chapter  6)  supports  the  claim  that  constructing  this  class  of  parallel¬ 
izer  is  not  only  possible,  but  also  can  be  useful. 

I  distinguish  these  three  levels  in  order  to  separate  the  effects  due  to  the  model 
itself  from  effects  due  to  specific  model-related  decisions  (like  granularity  or  choice 
of  transformations),  and  from,  accidental  effects  of  the  implementation.  The  interplay 
of  these  three  levels  is  easiev  to  understand  by  considering  an  example. 

First,  my  model  (the  higher  level)  recommends  that  we  represent  the  program 
as  a  hierarchy  of  pans,  and  that  our  compiler  ugh  the  hierarchy  in  the  order  of 

increasing  size.  When  dealing  with  some  recilit  language,  we  are  to  design  a 
repre.sentation  that  makes  this  hierarchical  u  er  ossible.  It  also  has  to  be  possi¬ 
ble  to  identify  promising  pans  of  the  program,  possibly  by  matching  them  against 
known  shapes. 

At  the  second  level,  we  choose  a  specific  representation  for  a  language  of 
interest.  For  instance,  we  bind  the  notion  of  granule  to,  say,  basic  blocks.  We  also 
have  to  define  fragments  somehow,  perhaps  we  choose  single-entry/single-exit 
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regions  of  the  flow  graph.  Now  we  have  to  decide  how  to  recognize  interesting 
shapes.  For  the  C  subset  used  in  this  investigation,  I  used  graph  parsing  with  respect 
to  the  SSFG  grammar  of  Farrow,  Kennedy  and  Zucconi  [FKZ76].  We  also  choose 
well-known  parallelizations,  to  show  that  they  can  be  e.xpressed  in  the  formalism  of 
my  model. 

At  the  bottom  level  we  have  the  decisions  that  enter  the  writing  of  the  specific 
parallelizer.  These  may  include  file  system  formats,  mode  of  interaction  with  the 
user,  etc.  For  SPOIL,  Chapter  6  provides  some  pertinent  detail. 

The  second  level  is  especially  important  for  this  thesis.  The  highest  level  is  so 
general  that  it  is  difficult  to  evaluate  its  relevance  by  means  of  testable  claims.  The 
bottom-most  level  is  so  crowded  with  implementation  details  that  it  is  hard  to  extract 
useful  generalizations  from  it  (that,  indeed,  is  the  characteristic  of  the  previous  work 
that  most  heavily  pressed  me  to  undertake  this  study).  It  is  at  the  second  level  that  1 
can  make  testable  claims  of  useful  generality.  I  choose  a  program  representation  by 
means  of  which  I  expect  to  encode  the  conditions  under  which  a  parallelization  is 
possible,  and  claim  that  such  representation  indeed  shows  existentially  that  the  gen¬ 
eral  model  is  applicable  in  a  practical  setting. 

1.4.  Previous  Work 

The  main  motivation  for  extracting  parallelism  from  ordinary  code  is  that  ordinary 
code  abounds.  After  the  advent  of  problem-oriented  programming  languages,  like 
FORTRAN,  most  computer  systems  presented  the  programmer  with  a  serial  mode  of 
operation,  where  the  concurrency  between  the  various  functional  units  of  the  com¬ 
puter  was  hidden.  And  thus  a  wealth  of  code  has  been  written  by  people  blissfully 
unaware  of  concurrent  execution  possibilities. 

Essentially  all  of  the  popular  problem-oriented  programming  languages  have 
been  defined  in  terms  of  sequential  semantics.  Thus,  whether  the  programmer 
intends  this  or  not,  there  exists  in  those  languages  a  well  established  order  of  execu¬ 
tion.  So,  the  availability  of  more  than  one  functional  unit  on  a  single  processor,  or  of 
several  processors  in  the  same  system,  is  not  necessarily  a  boon  to  the  execution  of 
such  programs:  it  is  not  immediately  clear  what  tasks  to  assign  to  anything  but  the 
first  such  functional  unit  or  processor. 

An  early  attempt  at  characterizing  the  conditions  under  which  one  can  relax  the 
default  semantics  was  reported  by  Bernstein  [Ber66].  His  three  conditions  (discussed 
later  in  Section  5.3.4)  for  independence  of  two  pieces  of  a  program  are  a  foundation 
for  the  work  that  followed.  First,  they  characterize  the  possible  independence  of  two 
parts  of  a  program  via  their  memory  reference  patterns.  Interfering  patterns  prevent 
parallel  execution.  Second,  the  conditions  leave  open  the  question  of  defining  what  a 
‘  ‘part’  ’  of  a  program  is. 

The  natural  inclination  then  w'as  to  consider  the  basic  blocks  of  the  program  as 
the  “parts”  in  the  conditions.  A  basic  block  is  a  list  of  instructions  that  contains  no 
branches,  except  perhaps  as  its  last  instruction,  and  no  jump  targets,  except  into  its 
first  instruction  (the  leader).  So,  once  control  reaches  the  leader,  all  the  other 
instructions  are  guaranteed  execution.  Conversely,  unless  the  leader  is  activated. 
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none  of  the  other  instructions  is.  This  all-or-none  property  makes  basic  blocks  an 
appealing  unit  of  decomposition  of  the  program. 

Basic  blocks  are  clearly  larger  (in  their  effect  on  the  store)  than  individual 
instructions.  However,  tb«'y  are  not  much  larger.  Tjaden  and  Flynn  [TjF70]  and 
Foster  and  Riseman  [FoR72]  noticed  that  there  isn’t  much  parallelism  available 
inside  a  typical  basic  block  (mainly  because  there  isn’t  much  going  on  in  a  typical 
basic  block).  So,  it  seems  one  would  prefe'.  to  look  at  coarser  partitions  of  the  pro¬ 
gram.  On  the  other  hand,  we  can’t  ignore  whatever  parallelism  is  present  at  (or 
under)  the  basic  block  level,  as  it  is  the  base  case  for  the  rest  of  the  analysis,  and  con¬ 
siderably  easier  than  the  across-block  detection  anyway.  Therefore,  all  the  paralleliz- 
ers  to  date  have  made  an  effort  to  enlarge  basic  blocks,  or  to  look  at  structures  that 
are  potentially  larger,  while  conserving  some  of  the  relative  simplicity  of  intra-block 
parallelization. 

Another  aspect  in  Bernstein’s  conditions  was  their  expression  in  terms  of  the 
state  of  the  store.  Two  sequential  “parts”  will  fail  to  be  parallelizable  if  the  effect  of 
the  sequence  of  reads  and  writes  to  the  store  determined  by  the  original  sequence  can¬ 
not  be  maintained  except  by  enforcing  the  original  sequence.  In  such  case,  one  says 
that  there  are  dependences  between  the  two  parts. 

The  first  successful  attempts  at  practical  parallelism  detection  were  based  on  a 
careful  study  of  those  dependences.  Kuck  [Ban79,  Kuc75]  and  his  students  and  col¬ 
leagues  at  University  of  Illinois  produced  Parafrase,  a  FORTRAN  compiler  which 
had  a  vectorization  facility.  A  new  generation  of  this  tool,  Parafrase-2,  is  described 
in  [P0I88]. 

Similar  work  was  pursued  elsewhere,  especially  at  Rice  University 
([A1K82,  ACK87,  Ken80],  and  IBM  T.J.Watson  ([ABC87]).  These  efforts  constitute 
the  recognized  mainstream  in  the  literature.  Their  main  common  point  is  the  use  of 
FORTRAN  as  their  target  language. 

Much  of  the  literature  of  the  late  70s  was  devoted  to  FORTRAN  on  vector 
machines.  Recently,  C  [KeR78,KeR89]  has  received  some  attention  too.  Parafrase-2 
is  expected  to  compile  both  Fortran  and  C.  And  Allen  and  Johnson  [A1J88]  describe 
a  C  compiler  for  Ardent’s  Titan. 

The  recognition  that  standard  programming  languages  may  make  the  recogni¬ 
tion  of  parallelism  difficult  (mainly  on  account  of  the  problem  of  resolving  aliased 
references  to  the  store)  has  induced  some  workers  to  consider  compilers  that  interact 
with  a  knowledgeable  user.  The  compiler  (actually,  a  programming  environment 
developed  around  a  compiler)  would  try  to  relax  dependences  as  far  as  possible,  and 
submit  to  the  human  user  the  consideration  of  those  dependences  that  appear  promis¬ 
ing  but  couldn’t  be  relaxed  automatically.  Examples  of  this  approach  are 
[ABC87,SmA88]. 

The  FORTRAN-centered  efforts  are  driven  by  the  desire  to  speed  up  scientific- 
code.  This  means  that  symbolic  computation  was  neglected  at  the  beginning,  as  this 
is  not  an  activity  for  which  FORTRAN  is  well  suited.  In  recent  times,  Scheme 
[Lar89]  and  Common  Lisp  [Har86,  PHK88]  have  received  some  attention. 
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While  iteration  is  the  natural  habitat  for  parallelism  in  FORTRAN  and  similar 
languages,  it  is  not  necessarily  so  for  Lisp.  So  one  can  expect  that  the  Lisp  paralleliz- 
ers  use  techniques  unusual  in  the  FORTRAN  world.  Still,  there  are  important  com¬ 
mon  points:  One  can  calculate  statically  a  reasonable  approximation  to  the  depen¬ 
dence  relationship  between  parts  of  the  program.  The  notion  of  dependence, 
explored  in  Chapter  2,  underlies  the  necessary  analyses.  For  the  difficulty  in  estab¬ 
lishing  accurate  control  flow  information  in  languages  like  Scheme,  one  can  refer  to 
[Shi88].  I  expect  my  model  to  cover  the  FORTRAN-in-LlSP  variety  of  numerical 
programs  as  easily  as  for  any  other  imperative  language.  In  this  sense,  it  should  be 
possible  to  reproduce  Harrison’s  [Har86]  results.  Further  work  is  needed  to  establish 
this,  and  to  explore  the  interaction  of  higher-level  imperative  features  (like  mapping) 
with  this  model  of  parallelization.  Interprocedural  analysis  is  needed  to  approach  the 
results  reported  by  Larus  [Lar89],  this  hasn’t  been  tried  yet  in  the  context  of  my 
model  but  is  a  natural  direction  for  extension. 

Another  dimension  of  variety  is  provided  by  the  hardware  used  to  run  parallel¬ 
ized  code.  For  VLIW  architectures  [Fis83],  there  is  J.  Ellis’s  Bulldog  compiler 
[E1186].  For  the  WARP  [AAG87,KuM84],  there  is  M.  L.am’s  compiler  [Lam89]. 
Finally,  for  data  flow  machines,  a  reference  of  choice  is  A.  Veen’s  SUMMER  com¬ 
piler  [Vee85]. 

All  these  (and  other)  proje  ts  have  produced  a  body  of  experience  on  how  to 
build  a  parallelizer.  As  mentioned  above,  one  motivation  for  the  work  reported  here 
was  the  absence  of  a  synthesis.  There  exist  many  specific  parallelizers,  but  little  in 
the  way  of  a  generalization.  In  terms  of  the  three-level  description  of  these  com¬ 
pilers,  one  can  say  that  most  of  the  literature  has  been  concerned  with  the  lowest  and 
second  lowest  levels:  specific  implementation  and  some  language-specific  theory. 

Recent  works  have  summarized  the  technical  aspects  of  this  experience 
[P0I88,  Wol89].  Reference  [P0I88,  Figure  1.3]  contains  a  model  for  Parafrase.  That 
model  distinguishes  high-  and  low-level  optimizations  (those  common  to  a  class  of 
architectures  vs.  those  specific  to  a  given  machine).  The  distinction  is  similar  to  the 
one  between  the  analyzer  and  the  synthesizer  in  my  model.  What  distinguishes  my 
model  is  (1)  the  usage  of  a  formally  described,  explicitly  parallel  representation  as 
the  interface  between  the  high-  and  low-level  parts  of  the  compiler,  (2)  the  treatment 
of  hierarchical  compositions  of  subparts  of  the  program,  and  (3)  the  encoding  of  the 
transformation  conditions  and  consequences  outside  the  compiler. 

1.5.  Plan  of  the  Dissertation 

Each  of  the  next  four  chapters  has  this  design:  First  1  discuss  the  theoretical  aspect  of 
the  model  that  supports  the  first  claim  of  the  thesis.  Then  I  consider  how  this  model 
has  been  implemented  in  SPOIL.  Then  I  discuss  some  related  work  and  the  implica¬ 
tions  of  this  model. 

Chapter  2  describes  the  input  demanded  by  the  parallelism  detector.  It  is  essen¬ 
tially  a  survey  of  the  state  of  the  art  in  analysis  of  sequential  programs. 

Chapter  3  describes  the  output  of  the  parallelism  detector.  A  mathematical  for¬ 
malization  of  the  notion  of  permissible  execution  orders  is  used  to  provide  a  meaning 
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for  the  transformations  that  the  detector  is  expected  to  perform. 

Chapter  4  describes  the  hierarchical  structure  of  program  fragments,  which  pro¬ 
vides  an  organized  search  space  for  the  parallelism  detector.  This  organization 
guarantees  that  the  results  of  parallelizing  part  of  a  program  may  be  used  when  try¬ 
ing  to  parallelize  a  larger  part.  It  also  provides  for  a  tradeoff  between  speed  of  com¬ 
pilation  and  thoroughness  of  the  parallelization. 

Chapter  5  deals  with  the  notion  of  program  transformation  that  is  relevant  to  this 
work.  By  separating  the  search  strategy  of  the  parallelizer  from  the  semantics  of  the 
objective  function,  this  model  permits  another  dimension  of  tradeoff  of  compilation 
speed  and  thoroughness  of  results. 

Chapter  6  summarizes  the  experience  gained  through  SPOIL.  Chapter  7  draws 
conclusions  about  the  generality  and  usefulness  of  this  model,  argues  for  the  thesis 
stated  above,  and  proposes  future  directions  of  work. 

Figure  1.3  shows  the  correspondence  between  chapters  of  this  work  and  parts  of 
the  model. 
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Figure  1.3.  Dissertation  Map 


CHAPTER  2 


Input  to  the  Parallelism  Detector 


The  goal  of  the  front  end  in  Figure  1.1  is  to  validate  the  syntax  and  semantics  of  the 
input  program,  and  then  to  provide  the  parallelizer  proper  with  a  convenient  represen¬ 
tation  of  the  original  program.  The  intention  here  is  to  abstract  the  essential  structure 
of  the  program  from  the  inessential  syntactic  details  of  the  source  code.  The  structure 
that  is  abstracted  consists  of  control  and  data  relationships  between  parts  of  the  pro¬ 
gram.  These  relationships  are  used  to  discover  the  dependences  that  constrain  our 
ability  to  execute  the  program  in  parallel.  In  what  follows,  the  assumed  representa¬ 
tion  is  a  control  flow  graph  with  data  flow  annotations. 

2.1.  Program  Graphs 

Consider  the  program  in  Figure  2.1.  The  purpose  of  that  code  is  to  compute  the 
minimum  and  maximum  values  of  the  array  x  (of  size  n),  and  return  them  via  the 
reference  variables  min  and  max. 

Assuming  that  no  errors  are  discovered  in  this  program,  the  front  end  will  pro¬ 
duce  some  internal  representation.  Figure  2.2  shows  an  approximation  to  the  three- 
address  code  (or  the  assembly  code)  that  a  classic  front  end  could  produce  from  the 
minmax  program. 

From  this  representation  one  can  construct  another  that  captures  better  the  way 
control  flows  through  the  program.  Figure  2.2  already  shows,  in  comments,  the 
boundaries  of  the  basic  blocks  of  minmax.  A  graph  whose  nodes  stand  for  the  basic 


proc  minmax  (array  x[n];  var  min, max); 
begin 

min  :=  max  :=  x[0]  ; 
for  int  i  =  1  to  n-1  do 

if  x[i]  <  min  then  min  :=  x[i]  end; 
if  x[i]  >  max  then  max  :=  x[i]  end; 

end; 

end. 


Figure  2. 1 .  Minimum  and  Maximum:  Code 
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#  START 


1- 

max  :=  x[0] 

# 

Basic 

Block 

2- 

min  :=  max 

3- 

i  :=  1 

4- 

$tqp  :=  n-1 

5- 

0: 

if  i  >=  $top  goto  3 

# 

Basic 

Block 

6- 

if  x[i]  >=  min  goto  1 

# 

Basic 

Block 

7- 

min  :=  x[i] 

# 

Basic 

Block 

8- 

1: 

if  x[i]  <=  max  goto  2 

# 

Basic 

B.lock 

9- 

max  ;=  x[i] 

# 

Basic 

Block 

10- 

2: 

i  :=  i-t-l 

# 

Basic 

Block 

11- 

goto  0 

12- 

3: 

return 

# 

STOP 

Figure  2.2.  Minimum  and  Maximum:  Assembly  Code 


blocks  and  which  has  an  arc  from  block  B  i  to  block  5  2  if  and  only  if  control  can  go 
from  lo  B  2  is  called  a  control  flow  graph  (or,  for  short,  graph).  For  more 
detail,  consult  [ASU86,  §10.4]. 

Instead  of  choosing  basic  blocks  as  the  nodes  of  flow  graphs,  one  could  choose 
the  3-address  statements  themselves,  or  some  other  unit  of  program  activity.  As  I 
would  like  to  separate  the  design  of  my  Analyzer  from  the  particular  choice  of  con¬ 
trol  flow  granularity,  I  introduce  the  notion  of  granule .  That  is  the  generic  name  that 
I  will  use  in  what  follows  to  describe  the  unit  of  analysis  resolution.  (This  is,  the 
nodes  of  the  flow  graph  obtained  from  the  front  end.) 

In  Chapter  3  I  develop  a  formalization  for  parallel  execution  of  ordinary  pro¬ 
grams.  Note  that,  in  that  formalization,  granules  serve  also  as  units  of  parallel  execu¬ 
tion. 

The  rest  of  this  section  is  a  quick  review  of  the  state  of  the  art  in  control  and 
data  flow  analysis.  Again,  [ASU86,  Chapter  10]  provides  a  convenient  reference. 

Figure  2.3  shows  a  flow  graph  for  miranax.  This  graph  captures  the  important 
aspects  of  the  way  control  flows  through  the  code.  For  instance,  we  can  infer  from 
the  graph  that  control  will  go  from  block  BB  3  to  55  4  when  the  comparison  x[i]  < 
min  is  true.  So  we  can  say  that  55  3  is  a  predecessor  of  55  4  (conversely,  that  55  4 
is  a  successor  of  55  3).  Notice  that  55  3  has  another  successor,  namely  55  5,  which 
is  also  the  successor  of  554.  Also,  55  3  is  preceded  by  55  2.  The  information 
about  55  3  and  55  4  can  be  summarized; 


succ  (55  3)  =  {554,  55  5} 
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Figure  2.3.  A  Basic-Blocks  Flow  Graph  for  Minmax 


pred{BB?,)={BB2} 

succ{BBA)=[BB5] 

pred{BBA)=[BB3} 

Similarly,  one  can  discover  that  there  are  paths  from  block  BB  2  to  each  of  the 
blocks  BBx,  for  j:  =  3,  4,  5,  6,  7\  Moreover,  all  paths  from  the  START  node  to  those 
other  blocks  pass  through  BB  2.  Consequently,  we  say  that  BB2  dominates  all  those 
other  blocks.  These  notions  of  succession  and  domination  are  examples  of  the  con¬ 
trol  flow  relations  that  can  be  discovered  from  the  flow  graph.  Their  discovery  is 
called  control  flow  analysis . 

Finally,  we  can  use  the  control  flow  relations  to  determine  other  properties  of 
the  program.  For  instance,  the  fact  that  BB2  dominates 
L  =  [BBx,  for  X  =  3,  4,  5,  6,  7}  and  that  control  only  leaves  L  to  go  to  BB2 
(succ(L)=  {BB2})  tells  us  that  {55  2)uL  is  a  loop,  whose  header  is  BB2  and 
whose  body  is  L. 

In  addition  to  the  control  flow  information,  one  can  also  analyze  how  data  is 
produced  and  consumed  by  the  various  parts  of  the  program.  For  instance,  we  can 
notice  that  block  BBl  both  uses  and  defines  a  value  for  i.  This  definition  of  i  kills 
the  one  in  fifi  1.  Because  of  the  loop  we  mentioned  above,  the  value  of  i  that  reaches 
block  BB  1  can  be  either  the  one  set  in  block  BB  1  or  the  one  set  in  a  previous  execu¬ 
tion  of  BBl.  If  we  number  each  definition  by  the  line  number  of  the  three-address 
statement  in  which  it  appears  (refer  to  Figure  2.2),  we  can  describe  what  we  just 
learned  about  BB  1  in  the  following  way: 

use{BBl)=[±] 
defmi)={^] 
kill{BBl)  =  [7>,  10) 
genmi)={l>,  10) 

This  information  exemplifies  the  data  flow  relations  in  the  program.  For  details 
of  the  data  flow  analysis  procedures,  see  [ASU86,  §10.5). 

2.2.  The  Flow-Attributes  Structure 

The  output  of  the  front  end  is  the  collection  of  control  and  data  flow  relations  it 
computes.  In  the  rest  of  this  work  it  is  useful  to  abstract  this  information  as  a  func¬ 
tion  that  associates  flow  information  with  a  piece  of  a  program. 


'  And  10  block  BB  8,  but  here  1  am  interested  only  in  the  flow  of  control  from  BB  2  into  the  loop  body. 
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I  am  going  to  represent  this  association  by  means  of  a  function  Flow  which  will 
take  a  granule  and  an  attribute ,  and  will  return  whatever  control  or  data  flow  infor¬ 
mation  is  associated  with  that  attribute  for  that  program.^ 

As  examples,  I  will  write: 

Flow{BB  3,  succ)  =  [BBA,  BBS} 

Flow(BB7,  use)  =  {/} 

In  the  examples  above,  the  first  arguments  have  been  basic  blocks.  Later  I  will 
generalize  this  usage  to  larger  units  of  code. 

2.3.  Related  Work 

The  information  contained  in  the  program  graph  can,  of  course,  be  rearranged  in 
ways  different  from  the  ones  I  postulated  in  the  previous  two  sections.  I  notice  here  a 
couple  of  variations  that  are  worth  keeping  in  mind. 

First  of  all,  the  fact  that  control  flow  is  represented  as  a  graph  means  that  some 
control  flow  relations  can  be  identified  by  detecting  patterns  in  graphs  (more  on  this 
later).  Of  course,  data  flow  relations  can  also  be  represented  as  graphs.  This  suggests 
trying  to  represent  the  data  flow  relations  in  the  same  graph  where  the  control  flow 
information  is  recorded.  See,  for  examples,  [AKP83],  or  fFe083],  or  [FOW87].  I 
keep  an  eye  open  on  the  possibility  of  using  one  of  these  representations  to  unify  the 
methods  described  later.  However,  there  aren’t  general  and  efficient  graph  matching 
techniques,  or  at  least  matching  techniques  that  are  efficient  for  the  combined 
representation.  On  the  other  hand,  the  SSFG  parser  is  well  suited  to  the  usual 
features  of  control  graphs,  and  it  is  reasonably  cheap,  being  linear  in  the  size  of  the 
graph  being  considered. 

Another  important  idea  is  that  some  blocks  compute  Boolean  values  that  ''ontrol 
the  flow  in  the  rest  of  the  program.  I  have  indicated  (by  means  of  a  question  mark 
after  the  controlling  expression)  the  conditions  computed  by  those  blocks  in  Figure 
2.3.  This  sort  of  control  dependence  information  can  be  computed  very  effectively 
independently  of  the  data  flow  computations  (see  [CyF88]  for  an  efficient  method). 


^  This  function  is  later  extended  to  fragments  and  aspects,  two  concepts  briefly  touched  upon  in  the 
inu-oduction,  and  defined  in  Chapter  4. 


CHAPTER  3 


Output  from  the  Parallelisin  Detector 


3.1.  Explicit  Parallelism 

The  dependences  provided  by  the  front  end  are  an  implicit  expression  of  the  available 
parallelism.  For  instance,  in  Figure  2.3,  we  could  discover  that  the  part  of  the  pro¬ 
gram  that  corresponds  to  blocks  BB1>  and  BBA  can  be  executed  in  any  order  with 
respect  to  the  part  composed  of  BB  5  and  BB  6.  (The  reason  being  that  the  minimum 
and  the  maximum  can  be  updated  independently.) 

However,  the  implicit  information  encoded  in  the  dependences  cannot,  by  itself, 
be  used  to  decouple  the  parallelism  detection  from  the  parallelism  exploitation. 
Indeed,  what  the  dependences  actually  tell  us  is  negative  information.  If  two  parts 
are  dependent,  then  there  is  no  parallelism  between  them.  Classical  dependence 
analysis  tells  us  explicitly  where  the  parallelism  isn’t,  but  the  synthesizer  of  Figure 
1.1  needs  to  know  where  the  parallelism  is.  Otherwise,  the  synthesizer  could  not  res¬ 
trict  itself  to  picking  between  parallel  representations;  it  would  have  to  construct 
them  on  demand  too. 

The  counterpan  to  this  observation  is  that  a  grossly  explicit  characterization  of 
the  available  parallelism  is  also  inadequate.  A  program  with  n  independent  and  indi¬ 
visible  parts  can  be  executed  in  n !  different  sequences,  whose  explicit  enumeration 
would  be  too  much  data  to  pass  around.  (Considering  that  this  example  is  the  best 
case  for  parallelization,  it  would  be  ironic  if  it  couldn’t  be  handled.)  So,  the  explicit 
parallelism  must  be  represented  in  sub-exponential  space,  if  the  separation  of  detec¬ 
tion  and  exploitation  is  to  be  practical.  I  will  refer  to  this  property  as  the  conciseness 
of  the  explicitly  parallel  repre.sentation. 

3.2.  Execution  Sets 

The  synthesizer  is  charged  w'*  producing  a  program  that  is  both  equivalent  to  the 
original  and  parallel,  and  for  this  it  will  use  the  explicitly  parallel  representation  that 
will  be  presented  soon.  This  section  formalizes  the  notion  of  safe  parallel  execution 
of  a  sequential  program.  The  view  here  is  static:  I  will  introduce  a  quantity  corre¬ 
lated  with  the  parallelizability  of  a  piece  of  code.  This  metric  captures  nicely  the 
notion  of  safe  parallel  execution.  I  will  formalize  in  this  chapter  the  notion  of  making 
a  piece  of  code  more  parallel  by  relating  it  to  the  increase  of  that  metric.  On  the 
other  hand,  I  won’t  explain  yet  how  to  go  from  a  less  parallel  to  a  more  parallel  situa¬ 
tion  (that  is  the  task  of  Chapter  5). 

Given  a  piece  of  code,  decomposed  in  several  parts,  one  can  entertain  the  idea 
of  executing  those  parts  in  various  sequential  orders.  As  a  first  example,  think  simply 
of  a  piece  of  code  decomposed  in  individual  instructions.  One  can  consider  what 
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happens  when  the  instructions  are  permuted  somehow  and  executed  in  a  sequence 
other  than  the  textual  one. 

Of  course,  most  permutations  of  a  program  are  either  nonsense  (because  illegal) 
or  a  different  program  altogether.  For  instance,  if  one  were  to  swap  the  initialization 
of  a  variable  with  its  first  use,  the  use  would  find  an  undefined  value  that  would  prob¬ 
ably  cause  a  failure  to  continue  executing,  or  would  produce  a  different  final  value. 

However,  there  are  some  permutations  that  leave  the  meaning  of  the  program 
untouched.  For  instance,  consider  this  piece  of  code: 

/*  il  */  X  =  X  +  1 

/*  i2  */  y  =  1 

Because  the  variables  used  and  modified  by  one  instruction  are  neither  used  nor 
modified  by  the  other,  it  should  be  clear  that  the  code  below  is  equivalent: 

/*  i2  */  y  =  1 

/*  il  */  X  =  X  +  1 


So,  the  sequential  execution  of  /'i  followed  by  22  is  equivalent  to  the  sequential 
execution  of  i  2  followed  by  2 1 .  This  observation  implies  that  the  parallel  execution 
of  2 1  and  2  2  is  also  equivalent  to  the  original  program.  Therefore,  statements  about 
sequential  executions  of  permutations  of  a  piece  of  code  can  be  used  to  derive  state¬ 
ments  about  parallel  executions. 

This  view  of  parallelism  is  not  new,  as  it  represents  properly  what  happens  in 
many  parallel  contexts:  there  is  some  common  clock,  or  a  level  of  atomic  operation, 
that  permits  one  to  interpret  parallelism  as  interleaving  of  sequential  executions. 
Equivalently,  a  legitimate  execution  of  a  parallel  program  has  the  same  effects  as 
some  sequential  execution  of  its  parts  (this  is  the  classic  notion  of  serializability). 
This  is  the  largely  implicit  assumption  in  the  early  work,  as  can  be  seen  in 
[Ber66,  Gil58,RaG69(Fig.2)].  Of  course,  this  assumption  is  generally  correct,  at  least 
for  closely  coupled  machines. 

The  advantage  of  making  this  assumption  is  that  it  enables  the  use  of  analysis 
technology  developed  for  the  sequential  case.  The  disadvantage  is  that  it  makes  it 
hard  to  study  the  case  of  processors  running  on  different  clocks  and  sharing  non- 
atomic  stores. 


The  connection  between  parallel  executions  and  interleavings  of  serial  execu¬ 
tions  makes  it  appealing  to  study  permutations  of  sequential  code.  (Actually,  I  need 
to  consider  permutations  of  subsets  of  sequential  code.)  To  begin  this  study,  consider 
a  set  M  and  a  partial  order  on  M.  Let  m  be  the  cardinality  of  M.  A  permutation  n 
of  M  is  an  injective  function  of  the  segment  of  the  naturals  {0,  1,  ...,  272-I}  onto  M. 
As  usual,  the  notation  will  represent  {0,  1,  ...,  m-1 }.  We  say  that  n  is  compati¬ 
ble  with  ‘(’  if  and  only  if 


(Vx,yeM)iVi,jeZJ 


ix=K{i)  A  y=n(j)  A x  <y)  =>  2  <j 


(3.1) 


Equation  3.1  says  that  ;t  enumerates  the  elements  of  M  in  a  sequence  compatible  with 
the  order 
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Now  consider  a  program  graph.  The  control  flow  and  data  flow  relations  com¬ 
puted  in  the  front  end  induce  a  partial  order  on  the  nodes  of  the  program.  (I  call  those 
nodes  the  mass  of  the  program).  That  order,  as  seen  by  the  front  end,  constrains  very 
rigidly  the  sequence  in  which  the  mass  can  be  executed.  Indeed,  the  front  end 
operates  under  the  assumption  that  the  code  will  be  executed  sequentially  from  begin¬ 
ning  to  end,  modulo  the  bypasses  produced  by  conditionals  and  the  repetitions  pro¬ 
duced  by  loops  (this  is  an  equivalent  version  of  the  hypothesis  of  ordinariness  of  the 
input  code). 

In  terms  of  order-compatibility,  the  front  end  assumes  that  only  one  permutation 
of  the  granules  is  compatible  with  both  the  control  and  data  flow  relations  associated 
with  the  program  graph,  once  input  is  provided  to  the  program.  In  the  itiinmax  exam¬ 
ple,  the  program  graph  indicates  the  following  order; 

BB\{BB2{BB3{BB4{BB5{BB6{BB1  (3.2) 

(where  x  (y  is  read  x  before  y)- 


If  we  consider  only  the  body  of  the  loop: 


6- 

if  x[i]  >=  min  goto  1 

#  Basic  Block  3 

7- 

min  ;=  x[i] 

#  Basic  Block  4 

8- 

1: 

if  x[i]  <=  max  goto  2 

#  Basic  Block  5 

9- 

max  :=  x[i] 

#  Basic  Block  6 

10- 

2: 

•  •  • 

we  can  observe  that  the  following  execution  sequences  are  possible  (in  the  sense  of 
respecting  the  partial  order  of  eq.  3.2): 

{BB  3,  BB4,BB5,BB  6) 

{BB  3,  BB  4,  BB  5) 

(55  3,55  5,55  6) 

(55  3,  55  5) 

Each  of  the  above  sequences  satisfies  the  constraints  imposed  by  the  order  of  eq. 
3.2.  Actually,  the  first  one  cannot  be  realized  from  any  input  data,  but  that  cannot  be 
discovered  by  looking  only  at  the  piece  of  program  above.  (The  invariant  min<max 
has  to  be  established  elsewhere.) 

We  have  already  noticed  that  the  combination  of  55  3  and  55  4  need  not  be  exe¬ 
cuted  before  the  combination  of  55  5  and  55  6  So,  in  addition  to  the  order  (eq.  3.2), 
we  already  know  that  this  order  is  also  permissible: 

55  1  <552<555<556<553  <554(557  (3.3) 

Once  this  is  noticed,  we  are  free  to  consider  a  larger  set  of  execution  sequences 
that  are  compatible  with  the  meaning  of  the  original  code: 

(55  3,  55  4,  55  5,  55  6) 

(55  3,55  4,55  5) 

(55  3,55  5,55  6) 

(55  3,55  5) 
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but  also 

{BB  5,  BB  6,  BB  3,  BB  4) 

(BB5,BB3,BB  4) 

(BB  5,  BB  6,  BB  3) 

(BB  5,  BB  3) 

The  execution  set  of  a  mass  of  code  with  a  partial  order  is  the  set  of  the  execu¬ 
tion  sequences  of  that  mass  that  are  compatib’e  with  the  partial  order. 

The  purpose  of  the  parallelism  detector  can  then  be  expressed  as  the  construc¬ 
tion  of  the  largest  (respectively,  least  restrictive)  execution  set  (respectively,  partial 
order)  for  the  mass  of  the  program.  This  is  the  promised  connection  between  a  metric 
(size  of  the  execution  set)  and  the  informal  notion  of  “degree  of  parallelizability’’. 

Individual  parallelizations  can  be  explained  this  way:  Given  an  execution  set  for 
a  program,  a  parallelization  is  a  transformation  that  yields  a  larger  execution  set. 
(Transformations  are  more  precisely  described  in  Chapter  5.)  A  parallelization  is 
called  safe  if  the  paths  in  the  new  execution  set  remain  compatible  with  the  depen¬ 
dences.  The  ideal  execution  set  for  a  program  (which  may  not  be  discoverable  in  the 
absence  of  perfect  information  about  the  program’s  inputs)  typically  includes  many 
more  paths  than  are  present  in  the  approximations  that  real  compilers  can  be  expected 
to  produce. 

Notice  the  following  interesting  property:  the  execution  set  of  a  program  is 
composed  of  sequential  enumerations  of  its  mass,  yet  its  size  gives  us  a  very  strong 
indication  of  potential  parallelism  in  that  mass.  This  is  so  because  the  pemiissibility 
of  parallel  executions  induces  many  compatible  sequential  executions.  So,  a  more 
parallel  interpretation  of  a  piece  of  code  necessarily  has  a  larger  execution  set. 
Although  the  converse  is  not  necessarily  true,  an  increase  in  execution  set  size  at  least 
makes  the  discovery  of  parallelism  more  likely. 

Finally,  a  note  on  terminology.  The  sequences  shown  above  (for  instance, 
(BB  4,  BB  5,  BB  3))  do  not  involve  all  the  mass  of  the  set  under  consideration.  Tech¬ 
nically,  they  are  called  selections  or  variations  on  that  mass,  the  name  pemiutations 
being  reserved  for  the  case  in  which  all  the  elements  are  involved.  This  distinction  is 
generally  immaterial  to  this  discussion,  where  for  convenience  I  use  the  term  permu¬ 
tation  as  shorthand  for  permutation  of  a  subset,  which  in  Combinatorics  would 
receive  one  of  the  names  mentioned  above. 

3.3.  Execution  Set  Notation 

Execution  sets  admit  a  concise  representation.  I  show  here  a  notation  that  expresses 
execution  sets  by  means  of  certain  symbolic  expressions  (S-expressions).  To  each 
such  execution  set  expression  corresponds  exactly  one  execution  set;  such  expres¬ 
sions  encode  exponential  numbers  of  execution  sequences  in  sub-exponential  space. 

The  simplest  execution  set  corresponds  to  individual  masses  of  code,  for  w'hich  I 
just  write  (execset  A«a.s-5-) . 

The  execution  set  corresponding  to  scheduling  several  others  in  sequential  order 
w'ill  be  represented  by  (series  execset^  •  •  •  execset^).  By  scheduling  in  sequential 
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order  I  mean  the  complete  execution  of  all  the  code  of  execseti  before  the  execution 
of  any  of  the  code  of  execsetj,  for  any  j  greater  than  i. 

The  execution  set  corresponding  to  scheduling  several  others  in  parallel  will  be 
represented  by  (parallel  execset\  execs etn) .  By  scheduling  in  parallel  I 
mean  that  the  code  associated  with  the  execution  sets  involved  can  be  interleaved  in 
any  way. 

Loops  will  be  represented  by  S-expressions  whose  heads  are  either  sloop  or 
ploqp,  designating  serial  and  parallel  loops,  respectively ^  The  tails  of  the  loop 
expressions  are  just  execsets,  implicitly  a  series. 

In  this  notation,  I  can  represent  the  execution  set  corresponding  to  execution 
orders  (3.2)  and  (3.3)  by  something  like 

(series  BB  1 

(sloop 

(parallel 

(seriesBB3BB4)  /update  min 
(series  5B  5  55  6))  /update  max 
BBl)) 

Notice  that  the  explicit  representation  of  the  parallelism  of  the  updates  of  min 
and  max  didn’t  add  an  exponential  amount  of  space  to  the  representation  of  just  those 
updates,  thanks  to  the  use  of  the  parallel  expression.  Indeed,  one  can  encode 
exponentially  large  numbers  of  permutations  just  by  giving  a  list  of  the  elements  (i.e., 
a  linear  amount  of  space  on  the  original  mass)  and  using  those  execset  operators. 

Finally,  the  serial  loop  above  can  be  distributed  over  the  two  independent  parts 
of  its  body,  which  can  be  represented  by: 

(parallel 

(sloop  55  3  55  4) 

(sloop  55  5  55  6) ) ) 

Now,  we  have  the  freedom  to  execute  the  loop  as  either  a  serai  loop  with  paral¬ 
lel  parts,  or  as  the  parallel  of  two  serial  loops.  This  sort  of  freedom  is  represented  by 
the  notation  (choice  execsetj  execseti  '''  execset n  ).  The  example  above 
yields: 

(series 
55  1 
(choice 
(sloop 
(parallel 


'  In  a  serial  loop,  no  iteration  begins  before  the  previous  one  is  finished.  In  a  parallel  loop,  no  iteration 
synchronizes  at  all  with  the  rest,  but  there  is  a  barrier  at  the  end  of  the  loop.  There  are  other  possible 
intcrpretauons  for  parallel  loops  (e.g.,  iterations  may  be  allowed  to  overlap  in  specified  ways),  but  they 
can  be  expressed  using  just  series,  parallel,  sloop,  and  ploop. 


22 


(series  55  3  55  4)  /update  min 
(series  55  5  55  6) )  /update  max 
(parallel 

(sloop  55  3  554) 

(sloop  55  5  55  6) ) ) 

BBl)) 

To  say  that  the  execution  set  of  a  piece  of  code  is  a  choice  of  other  execution 
sets  means  that  to  execute  the  piece  of  code  successfully,  it  is  necessary  to  execute 
exactly  one  of  the  choices. 

3.4.  A  Calculus  of  Execution  Set  Composition 

An  important  feature  of  the  execset  notation  is  that  it  is  possible  to  calculate  with  it. 
In  particular,  considerations  of  inclusion  or  relative  size  of  execution  sets  can  be 
made  without  reference  to  the  elements  of  the  sets  themselves. 

For  instance,  assume  that  you  are  given  the  execset  expression  (series  A 
(series  B  ...)).  This  expression  represents  those  permutations  of  {A  B,...}  that 
are  series  in  which  first  A  is  executed,  and  then  a  series  is  executed,  consisting  of  B 
and  the  rest.  Well,  we  can  reason  that,  independently  of  A  B,  etc.,  the  expression 
above  is  nothing  but  the  series  that  involves  all  those  masses  in  the  same  order.  We 
can  represent  this  conclusion  by  a  simple  rule: 

(series  A  (series  B  . . . ) ) 

<=> 

(series  A  B  . . . ) 

We  can  use  this  rule  to  simplify  an  execset  expression  that  contains  that  nested 
usage  of  series.  Although  this  rule  is  established  by  consideration  of  the  meanings 
of  the  execution  set  expressions  involved,  its  application  is  completely  syntactic  in 
nature;  once  the  rule  is  established,  it  can  be  used  as  a  lemma  in  future 
simplifications,  without  having  to  resort  again  to  analyzing  the  meaning  of  the  partic¬ 
ular  expressions  involved. 

3.4.1.  Semantics  of  Execution  Set  Notation 

Here  I  give  a  more  formal  presentation  of  the  semantics  behind  the  Execution  Set 
notation.  The  intention  is  to  justify  the  mechanics  of  the  calculus. 

3.4.1. 1.  Paths 

Let  S  be  a  finite  set  {5o.  ■S'  i .  •••»  )  •  A  path  through  5  is  a  permutation  of  a  subset  of 

S.  More  precisely,  if 

{3k<n)  p  =  (Si^^  5,-j  ...,  Si^) 

where  each  5, vE 5,  with  no  repetitions.  ^  ^ 

then  pe  Paths  (S). 

I  now  define  some  operations  that  construct  paths  from  paths.  Let  and  5  2  be 
finite  sets,  with  typical  elements  sn  and  S2i,  respectively.  Let  also 
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Pi  ={su),  \<i<ki^ki<card{Si) 


and  P2  =  (‘^2;)-  l^j^k2,k2^card{S2)  be  paths  through  5]  and  S2- 

The  disjoint  sum  of  pi  and  P2  is  the  concatenation  of  both,  in  the  same  order, 
and  is  defined  by: 

Pi©P2=(‘yii,  •••.‘^^,,‘^21.  •••^S2k2)  (3-5) 


The  disjoint  suir.  of  two  paths  happens  to  be  a  path  too:  it  is  clear  that  pi©P2 
belongs  in  the  paths  of  the  disjoint  union  of  5]  and  S2  (whence  the  qualification  of 
disjoint  sum).  Other  properties:  The  empty  sequence  is  an  identity  for  this  operation. 
Disjoint  sums  are  associative  (but  not  commutative). 

The  definition  extends  to  sets  of  paths:  the  disjoint  sum  of  two  sets  of  paths  is 
the  set  of  the  disjoint  sums  of  pairs  of  paths,  taken  one  from  each  of  the  given  sets. 

A  disjoint  product  of  pi  and  P2  is  any  proper  interleaving  of  their  pans,  as 
defined  by: 


Pi<S)p2  =  (^1,  t2,  h^+k2) 

where  tk  =  5,7<=>(Vg,  1^<7)  (3/i,  l<h^)\th  =  Sig 


(3.6) 


Although  there  is  ort] 

,  .  ,,  |^i+^2i 

products  (actually. 


y  one  disjoint  sum  of  two  paths,  there  are  many  disjoint 
in  total). 


The  empty  sequence  is  again  an  identity  for  this  operation 

The  definition  again  extends  to  sets  of  paths.  The  disjoint  product  of  two  sets  of 
paths  is  the  set  of  the  disjoint  products  of  pairs  of  paths,  taken  one  from  each  of  the 
given  sets. 

Most  uses  of  disjoint  products  in  what  follows  refer  to  the  properties  of  the  set 
extension.  So,  a  bit  of  notation  is  required;  when  in  a  disjoint  product  I  use  the  name 
of  a  path,  it  should  be  understood  that  the  singleton  set  of  that  path  is  actually  used. 
So,  in  what  follows,  I  take  disjoint  sums  and  products  to  be  defined  on  sets  of  paths 
and  yielding  sets  of  paths. 


The  proof  of  the  following  theorem  demonstrates  these  conventions. 


Theorem  3.1 

Disjoint  products  are  associative. 

Proof. 

Consider  p®{q<S)r).  A  representative  of  this  set  of  paths  will  be  a  path  that  con¬ 
tains  elements  from  p,  q,  and  r.  Define  a  mapping  from  sequences  to  sequences 
that,  given  the  representative  chosen  above,  will  return  a  sequence,  call  it  pq, 
that  ignores  the  elements  of  r.  Notice  that  (1)  this  sequence  is  a  path,  indeed  an 
element  of  p®q,  and  (2)  the  representative  sequence  is  a  result  of  interleaving 
the  elements  of  r  into  pq.  Thus,  the  representative  belongs  in  ip®q)®r  too. 
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The  other  inclusion  mirrors  this  argument. 

□ 

Disjoint  products  and  sums  do  not  distribute.  Instead,  we  have 

Theorem  3.2 

((p®^)0r)u  (q®(p®r))  is  a  proper  subset  of  p®{q®r) 

Proof. 

We  prove  this  for  the  case  of  singleton  arguments,  as  that  is  enough. 

Every  element  of  the  right  hand  side  can  be  seen  as  the  interleaving  of  elements 
taken  from  p  into  a  concatenation  of  q  and  r.  There  are  three  classes  of  distribu¬ 
tions  of  the  elements  of  p  into  the  concatenation.  First  of  all,  the  elements  of  p 
could  be  interleaved  among  the  elements  of  q,  leaving  the  elements  of  r  uninter¬ 
rupted  at  the  tail  of  the  interleaving.  These  distributions  are  all  members  of 
{p®q)®r.  A  second  class  leaves  the  elements  of  q  contiguous,  but  interleaves 
all  the  elements  of  p  among  the  elements  of  r.  These  distributions  are  all 
members  of  q®(p®r). 

The  elements  in  either  of  the  above  classes  (i.e.,  in  their  union)  are  all  interleav¬ 
ings  ofp  into  the  concatenation  of  q  and  r.  This  shows  that  the  left  hand  side  is 
a  subset  of  the  right  hand  side. 

On  the  other  hand,  there  are  interleavings  of  p  into  q®r  that  leave  some  ele¬ 
ments  of  p  among  the  elements  of  <?,  and  the  rest  among  the  elements  of  r. 
Those  interleavings  form  the  third  class,  which  is  non-empty  when  p,  q  and  r  are 
paths  with  more  than  one  element.  Thus  the  inclusion  is  proper. 

□ 

Notice  that  the  disjoint  sum  of  two  paths  is  just  another  of  their  interleavings. 
So,  when  dealing  with  sets  of  paths  A  and  B,A®B  is  a  proper  subset  of  A®B. 

If  all  the  sets  involved  are  the  same,  the  disjoint  sum  and  product  have  a  espe¬ 
cially  simple  form  that  deserves  its  own  notation.  The  scalar  multiple  of  a  path  p  by 
the  natural  number  k  is  defined  by: 

0-p  =  0 

(3  7) 

k-p  =p®{{k-\yp) 

Similarly,  a  ^th  power  of  a  path  is  defined  by: 

/)"=() 

These  definitions  extend  to  sets  of  paths  too,  in  the  natural  way.  In  that  exten¬ 
sion  we  see  also  that  the  ^th  multiple  of  a  set  is  a  subset  of  the  /<:th  power  of  the  same 
set. 
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3.4.I.2.  Execution  Set  Meanings 

We  can  use  the  disjoint  sum  and  product  operations  to  define  formally  what  each  of 
the  execution  set  forms  mean. 

execset 

The  form  (execset  /)  stands  for  the  subset  of  Paths  {Mass  if))  that  is  compati¬ 
ble  with  the  flow  dependences.  This  form  is  called  a  trivial  execset,  as  it  does 
not  encode  any  information  beyond  the  definition  of  execution  set.  It  is  there¬ 
fore  a  base  case  for  structural  induction  when  dealing  with  execsets. 

series 

The  form  (series  e\  62  •  •  ‘ )  stands  for  the  disjoint  sum  of  the  sets  defined 

by  Cl,  ^2,  •  •  •  • 

peurallel 

The  form  (parallel  e\  62  •  • '  )  stands  for  the  disjoint  product  of  the  sets 
defined  by  Ci,  C2,  •  •  •  • 

sloop 

t:=oo 

(sloop  e)=  u  k  c  (3.9) 

k=Q 


ploqp 


(ploop  e)  =  u 


(3.10) 


choice 

A  choice  between  execsets  stands  for  the  (disjoint)  union  of  the  sets  represented 
by  its  subforms. 

3.4. 1.3,  Simplification  by  identities 

These  definitions  can  be  used  to  prove  the  validity  of  a  collection  of  simplification 
identities,  as  well  as  to  guide  the  discovery  of  other  relationships.  For  instance,  the 
simplification  of  (series  A  (series  B  C))  to  (series  ABC)  is  a  consequence 
of  the  associativity  of  the  disjoint  sum.  Also,  the  fact  that  the  execution  set  denoted 
by  (parallel  A  (series  B  C))  is  larger  than  the  one  of  (choice  (series 
(parallel  A  B)  C)  (series  B  (parallel  AC)))  is  a  consequence  of  the  lim¬ 
ited  distributivity  of  the  disjoint  product  into  the  disjoint  sum. 

In  presenting  these  identities,  I  will  write  x  .  y  whenever  I  mean  the  list  whose 
first  element  is  x  and  whose  rest  is  the  list  y,  (i.e.,  (cons  x  y)),  and  x  |  y  when¬ 
ever  I  mean  the  list  whose  elements  come  from  the  lists  x  and  y,  listed  in  that  order, 
i.e.,  (append  x  y) .  Also,  variables  like  ?x  stand  for  anything  that  could  match  part 
of  a  pattern. 

Trivial  Nests 

Nl:  (series  ?A)  <=>  ?A 
N2:  (parallel  ?A)  <=>  ?A 
N3:  (choice  ?A)  «  ?A 
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These  are  especially  obvious  identities.  Essentially,  if  previous  simplifications  have 
reduced  a  expression  to  a  condition  in  which  there  is  only  one  thing  to  do,  that  thing 

gets  done  without  further  qualification.  Of  course,  this  doesn’t  apply  to  loops,  as  in  a 

loop  there  may  be  only  thing  to  do,  but  it  will  be  done  possibly  more  than  once. 

Loops 

LI;  (sloop  (series  .  ?A))  <=>  (sloop  .  ?A) 

L2:  (plocp  (series  .  ?A) )  <=>  (plocp  .  ?A) 

These  depend  on  noticing  that  the  bodies  of  loops  are  series  by  definition. 
Non-trivial  Nests 

01;  (series  ?X) | ((series  .  ?Y)  .  ?Z))  o  (series  ?X|?Y|?Z) 

02;  (parallel  ?X)  |  ((parallel  .  ?Y)  .  ?Z) )  <=>  (parallel  ?X|?Y|?Z) 
03;  (choice  ?X) | ((choice  .  ?Y)  .  ?Z))  o  (choice  ?X|?Y|?Z) 

These  are  generalizations  of  the  example  at  the  beginning  of  this  section.  Essentially, 
if  a  series,  parallel,  or  choice  expression  is  immediately  embedded  into  another, 
the  inner  one  can  be  spliced  into  the  outer  one  without  damage.  These  generaliza¬ 
tions  follow,  respectively,  out  of  the  associativity  of  the  disjoint  sum,  the  associa¬ 
tivity  of  the  disjoint  product,  and  the  associativity  of  the  disjunction  in  propositional 
logic. 

3.4.I.4.  Simplification  by  inequalities 

The  results  of  Section  3  4.1.3  apply  to  the  simplification  of  an  execution  set  expres¬ 
sion  to  another  that  stands  for  exactly  the  same  execution  set.  There  are  also 
simplifications  that  are  possible  only  when  an  execution  set  is  known  to  be  a  proper 
subset  of  another. 

First  we  define  a  partial  order  on  execution  set  expressions,  namely  the  order 
induced  by  inclusion  among  execution  sets.  Let  and  esi  be  two  execution  set 
expressions,  and  let  E\  and  Ej  be  the  execution  sets  they  stand  for.  Then  eS2  sub¬ 
sumes  if  and  only  if  Ei  is  a  proper  subset  of  £'2-  This  I  write;  esi  <  es2- 

As  examples,  notice  that  the  observation  that  /tth  multiples  are  subsets  of  kxh 
powers  means  directly  that  the  execution  set  represented  by  (sloop  (execset  A') ) 
has  to  be  smaller  than  the  execution  set  represented  by  (ploop  (execset  X) ) . 
Thus,  for  all  X,  we  can  write: 

(sloop  (execset  X) )  <  (ploop  (execset  X) ) 

and,  with  more  reason 

(series  .  ?X)  <  (parallel  .  ?X) 

These  inequalities  in  execution  set  expressions  find  application  in  the  elimina¬ 
tion  of  redundant  branches  in  choices.  A  branch  in  a  choice  is  redundant  if  and 
only  if  it  is  subsumed  by  another  branch.  Removing  a  redundant  branch  does  not 
change  the  execution  set  represented  by  the  choice  expression. 

For  instance,  assume  that  an  execution  set  is  represented  by  a  choice  between 

(sloop  (ploop  (execset  1) ) (sloop  (execset  2) ) )  and  (sloop  (ploop 
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(execset  1) )  (ploop  (execset  2) ) ) .  As  the  execution  set  (sloop  (execset 
2))  is  just  a  subset  of  (ploop  (execset  2)),  there  is  no  damage  in  discarding 
completely  the  first  interpretation. 

So,  we  have  two  more  identities: 

PI:  (choice  (series  .  ?X)  (parallel  .  ?X)  .  ?y) 

<=>  (choice  (parallel  .  ?X)  .  ?¥) 

P2:  (choice  (sloop  .  ?X)  (ploop  .  ?X)  .  ?Y) 

<=>  (choice  (ploop  •  ?X)  .  ?y) 

In  PI  and  P2,  the  rules  hold  also  if  the  order  of  the  series  and  parallel  forms  is  per¬ 
muted,  of  course.  More  generally,  the  order  of  execution  sets  that  are  parts  of  a 
choice  expression  is  irrelevant,  and  so  we  can  reorder  tliose  expressions  if  so  desired. 

3.4.I.5.  Reduced  Forms 

The  identities  and  inequalities  of  Sections  3.4. 1.3  and  3.4. 1.4  show  that  a  given  exe¬ 
cution  set  can  be  represented  by  many  equivalent  expressions,  the  equivalence  arising 
essentially  from  syntactic  redundancy".  In  order  to  permit  efficient  comparison  of 
execution  sets,  their  expressions  should  be  stored  in  a  common  form  that  does  not 
suffer  from  this  redundancy.  (The  Transformer  of  Chapter  5  will  have  abundant  need 
to  compare  execution  sets.) 

To  this  effect,  I  will  use  the  identities  and  inequalities  described  above  to  define 
transformations  of  execution  set  expressions.  The  transfonoations  will  be  named 
a^ter  the  identities  and  inequalities  themselves:  Nl,  N2,  N3,  LI,  L2,  01,  02,  03, 
PI,  and  P2.  In  each  case,  the  transformation  is  defined  this  way:  an  execution  set 
expression  that  matches  the  left  hand  side  of  a  transformation  can  be  replaced  with 
the  right  hand  side,  properly  instantiated  to  reflect  the  match. 

I  say  that  an  execution  set  expression  is  in  its  reduced  form  when  recursive 
application  of  the  transformations  Nl,  N2,  N3,  LI,  L2,  01,  02,  03,  PI,  and  P2  can¬ 
not  modify  it  any  further.  Let’s  refer  to  this  process  as  Algorithm  R . 

Also  as  the  order  of  branches  in  a  choice  is  irrelevant,  I  assume  that  transforma¬ 
tions  PI  and  P2  are  free  to  reorder  their  choice  patterns  in  order  to  establish  the 
match. 


^  This  redundancy  is  almost  trivial,  as  it  can  be  judged  by  inspection  of  the  execution  set  expressions 
alone.  On  the  other  hand,  the  redundancies  discussed  in  Section  5.5.1  can  be  detected  only  by 
considering  the  arrangement  of  parts  of  the  specific  program  at  hand,  and  therefore  demand  a 
comparison  of  the  meanings  of  the  execution  set  expressions,  in  the  context  of  a  given  program.  Thus 
the  distinction  be'ween  syntactic  simplification  (which  involves  only  the  forms  of  the  expressions)  and 
semantic  simplification  (which  demands  the  meanings  of  the  expressions). 
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Theorem  3.3 

Each  execution  set  expression  has  a  unique  reduced  form,  except  for  the  order 
of  choice  branches. 

Proof. 

First  of  all,  let’s  observe  that  Algorithm  R  is  indeed  an  algorithm.  Given  an  exe¬ 
cution  set  expression  e,  each  of  the  transformations  Nl,  N2,  N3,  LI,  L2,  01, 
02,  03,  PI,  and  P2  reduces  the  number  of  parentheses  in  e  by  exactly  2.  So, 
either  e  is  converted  into  an  expression  with  fewer  parentheses,  or  it  is  a  hxed 
point  of  the  transformation  set.  In  the  former  case.  Algorithm  R  cannot  iterate 
indefinitely,  as  the  number  of  parenthesis  pairs  would  reduce  at  each  iteration. 

Now,  observe  (by  case  analysis)  that  when  any  two  of  these  rules  apply, 
the  application  of  either  does  not  preclude  the  (eventual)  application  of  the 
other.  So,  we  are  free  to  apply  them  in  any  sequence,  and  in  as  much  as  we 
keep  doing  it  while  possible,  the  resulting  execution  set  expressions  will  be  the 
same.  (Except,  perhaps,  for  the  irrelevant  order  of  branches  of  a  choice).  This 
is,  the  set  of  transformations  supplied  to  Algorithm  R  has  the  Church-Rosser 
property. 

Thus,  the  procedure  defined  by  Algorithm  R  terminates  for  any  execution 
set  expression  e  by  yielding  a  possibly  different  execution  set  expression  r.  and  r 
is  unique  up  to  the  order  of  choice  branches. 

□ 

3.5.  Discussion 

The  motivation  behind  the  development  of  the  execution  set  calculus  is  simple;  to 
represent  abundant  parallelism  compactly,  and  in  a  way  that  can  be  manipulated  sym¬ 
bolically  at  low  cost. 

Having  related  execution  set  size  with  parallelizability,  I  will  show  later  how 
one  can  structure  the  parallelism  detector  around  the  theme  of  enlarging  execution 
sets.  The  application  of  the  theory  presented  in  this  chapter  will  be  seen  in  the 
Transformer  of  Chapter  5. 

As  I  said  above,  the  motivation  for  this  development  was  the  efficient  symbolic 
manipulation  of  compact  parallelism  representations.  These  desires  can  conflict.  For 
instance,  simple  regular  expressions  can  be  used  to  represent  flow  structures  (for 
i'.'itance,  as  in  Path  Expressions  [CaH74,Cam76],  or  in  the  work  of  Wegman 
[Weg83]).  Adding  to  this  notation  the  closure  of  shuffles  [Jed87]  increases  the  power 
of  the  notation,  but  also  increases  the  recognition  (and,  therefore,  the  manipulation) 
complexity  of  the  generated  language  to  context  sensitive  [Jed88].  Further  adding 
explicit  locks  to  represent  arbitrary  synchronization  provides  a  very  detailed 
representation  of  parallelism,  but  at  the  cost  of  raising  the  complexity  of  the  language 
generated  by  these  expressions  all  the  way  to  recursively  enumerable  [Sha78]. 

Execution  set  expressions  strike  a  balance,  in  remaining  easy  to  manipulate 
while  providing  detail  that  parallelizers  need.  Nested  ploops  cause  them  to  generate 
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a  hierarchy  of  context  sensitive  languages  (the  argument  follows  those  of 
[Jed87,  Jed88]  with  respect  to  general  shuffle  operations),  but  the  resulting  power 
doesn’t  prevent  us  from  calculating  with  the  resulting  expressions. 

In  reaching  this  balance,  some  detail  is  lost.  For  instance,  execution  set  expres¬ 
sions  don’t  represent  conditionals,  nor  do  they  represent  explicit  synchronization,  nor 
can  they  be  used  to  represent  certain  architectural  constraints  (like  size  of  the  vector 
register  file  in  vector  machines). 

Some  of  this  detail  has  been  dispensed  with  because  it  can  be  recovered  at  will 
from  other  information  available  to  the  compiler.  For  instance,  conditionals  could 
map  nicely  to  alternation  operators  (as  in  Flow  Expressions  [Sha78],  for  instance), 
but  I  had  a  more  useful  employment  for  alternation,  in  the  choice  operator.  That 
doesn’t  prevent  the  parallelizer,  as  will  be  seen  later,  from  using  knowledge  about 
conditionals  in  parallelization:  conditionals  can  be  detected  and  recovered  easily 
from  the  flow  graph,  which  is  part  of  the  assumed  input  to  the  parallelizer. 

The  lack  of  explicit  synchronization  comes  from  the  desire  to  avoid  committing 
to  a  specific  synchronization  mechanism  (say,  semaphores),  as  this  would  bring  in  too 
much  architectural  detail  to  the  analysis.  However,  there  is  no  way  to  avoid 
representing  some  synchronization,  and  it  is  done  implicitly  at  the  end  of  parallel 
and  ploop  operations. 

Similarly,  the  lack  of  mechanisms  to  represent  some  architectural  constraints  on 
parallelism  is  necessary  to  avoid  contaminating  the  detector  with  exploitation  issues. 
For  instance,  vectorizations  have  the  form  (ploop  <expression>) ,  which  does  not 
show  how  many  vector  registers  are  effectively  usable.  This  is  a  runtime  feature,  best 
left  to  the  back  end. 

Finally,  in  spite  of  my  best  wishes  to  avoid  committing  to  detail  too  early,  some 
model  of  parallel  execution  is  required  behind  the  syntax.  The  execution  set  expres¬ 
sions  embrace  the  well-known  assimilation  of  parallelism  to  arbitrary  interleaving  of 
sequential  actions. 

3.6.  Related  Work 

Describing  parallelism  explicitly,  and  trj'ing  to  quantify  what  it  means  for  a  piece  of 
code  to  be  ‘more  parallel’  than  another,  is  central  to  any  systematic  parallelization 
efforts.  So,  every  parallelization  project  has  had  to  define  a  measure  of  increasing 
parallelism. 

While,  in  most  cases,  the  metric  is  only  implied  by  the  rest  of  the  work  (for 
instance,  the  number  of  dependences  carried  by  a  loop  can  be  used  to  show  that  a 
transformation  exposed  some  parallelism),  some  attempts  have  tried  to  describe  for¬ 
mally  what  it  means  for  an  algorithm  to  be  ‘more  parallel’. 

For  instance,  Amir  and  Smith  [AmS86]  discuss  parallelism  in  CREW  models. 
There,  nesting  of  composition  is  related  to  synchronization,  and  therefore  shallower 
nesting  means  more  chances  for  parallelism.  Their  work  is  restricted  to  the  primitive 
recursive  functions. 
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Shaw  [Sha78]  describes  a  family  of  languages  based  on  regular  expression 
abstractions  of  flows  (of  data,  of  control,  ...)  in  systems.  His  Flow  Expressions  are 
very  powerful  (every  recursively  enumerable  set  can  be  encoded  as  one),  and  thus 
they  may  be  hard  to  manipulate  symbolically.  Execution  Set  expressions  can  be  con¬ 
sidered  a  very  restricted  subset  of  Flow  Expressions,  chosen  for  fit  to  the  task  at  hand 
and  ease  of  manipulation.  In  addition.  Execution  Set  expressions  map  alternation  (in 
regular  expressions)  to  choice  (don’t-care  non-determinism),  while  Flow  Expressions 
normally  interpret  alternation  as  a  representation  for  conditionals.  Flow  Expressions 
distinguish  finite  iteration  (possibly  unbounded)  from  guaranteed  endless  looping. 
This  distinction  adds  complexity  and  would  not  be  very  fruitful  in  the  context  of  this 
work. 

The  disjoint  product  is  essentially  equivalent  to  the  shuffle  operator.  A  good 
survey  can  be  found  starting  from  [Jed87,  Jed88]  and  the  references  therein. 

Milne  [Mil85]  describes  a  mathematical  framework  to  analyze  concurrent  sys¬ 
tems.  His  system  is  non-detemiinistic.  His  contribution  is  a  formal  calculus  of  con- 
cun  ency. 

Time,  and  concurrency,  can  be  given  formalizations  in  logic.  See,  for  instance 
[Gol87]. 

Finally,  Lamport  [Lam86]  has  studied  formalizations  of  parallelism  that  do  not 
demand  serializability,  nor  even  atomicity.  Perhaps  a  different  parallelization  model 
could  be  designed  around  these  ideas,  but  it  would  likely  require  a  different  analysis 
strategy.  (As  I  said  above,  the  advantage  of  reducing  parallelism  to  interleaving  is 
that  this  reduction  permits  the  use  of  classical  techniques  for  the  analysis  of  sequen¬ 
tial  programs.) 


CHAPTER  4 


Bottom-Up  Structure:  The  Fragment  Enumerator 


So  far  we  have  tools  to  characterize  our  freedom  to  schedule  pieces  of  a  program  in 
different  orders.  However,  we  have  no  notion  of  what  pieces  of  a  program  it  could  be 
interesting  to  analyze. 

Choosing  the  program  decomposition  properly  is  critical.  First  of  all,  there  are 
far  too  many  subsets  of  a  program  for  one  to  consider  seriously  exploring  all,  or  even 
a  large  number,  of  them.  Second,  not  all  subsets  of  a  program  are  ‘interesting’.  The 
notion  of  interest  here  is  twofold:  (1)  selecting  a  subset  of  the  program  is  a  form  of 
abstraction,  some  abstractions  capture  the  meaning  of  the  program  more  usefully  than 
others;  (2)  empirically,  we  see  that  potential  for  parallelism  is  ofm  found  in  some 
places  (for  example,  loops),  which  suggests  that  special  effort  be  dedicated  to  their 
analysis 

The  goal  here  is  to  find  a  definition  of  fragment  that  satisfies  as  much  as  possible 
conditions  like  (l)-(3)  of  §1.2  and  for  which  the  computation  of  flow  attributes  can 
be  accomplished  efficiently. 

There  are  many  conceivable  definitions  of  fragment.  Compilation  technology 
has  already  accumulated  some  experience  with  various  classes  of  regions,  like  inter¬ 
vals  [A1C76,  HeU72,Tar74]  ([Bur87]  provides  a  nice  summary  of  the  state  of  the  art), 
hammocks  [Kas75,  Kas73]  and  other  similar  notions  [Sha80,Wad80].  I  have  selected 
depth-first-numbering  intervals  as  ,  ny  definition  of  fragment,  and  examples  from  here 
on  will  assume  it.  Hov'ever,  it  is  important  to  distinguish  the  general  notion  of  frag¬ 
ment  from  its  instantiations.  The  model  works  in  terms  af  fragments,  that  nest  to 
reflect  the  semantic  T  the  source  language.  Practically  all  the  details  that  follow  can 
be  transcribed  mechan-  .ally  to  use  some  other  definition  of  fragment. 

4.1.  Dopth-First-Numbcring  Intervals 

Given  a  directed  graph,  its  nodes  can  be  numbered  in  the  sequence  of  their  preorder 
visit,  starting  from  some  root  node.  These  numberings  are  called  depth-first  number¬ 
ings ,  because  they  correspond  to  the  order  in  which  the  graph  would  be  visited  dur¬ 
ing  a  depth-first  search.  (The  standard  reference  to  why  depth-first  numberings  are 
useful  is  [HeU75])  I  will  use  the  abbreviation  dfn  to  refer  to  such  numberings,  so 
dfn(i)  refers  to  the  node  whose  depth-first  number  is  i  and  dfn~^(p)  refers  to  the 
depth-first  number  of  the  node  named  p. 

Consider  a  graph  G  =(V,  E)  with  n  nodes,  and  think  of  the  nodes  whose  dfn 
numbers  k  belong  in  the  interval  0<i<k<j<n.  For  some  choices  of  i  and  j  it  will  be 
true  that  that  set  is  closed  with  respect  to  the  original  graph,  in  the  sense  that 
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(V /;  ^.r.  0</</:<y'<n)  (V/J  e  V)  \^{dfn{k),  p)e  E  i<dfn  ^(p)<yj  (4.1) 

In  this  case,  we  call  that  set  a  depth-first-numbering  interval,  or  dfn-interval  for 
short.  A  dfn-interval  is  notated  by  listing  its  extreme  depth-first  numbers  separated 
by  a  semicolon,  thus:  i  ;j.  Such  interval  is  said  to  be  of  size  j-i. 

The  intuitively  important  characteristic  of  dfn-intervals  is  that  all  the  arcs  that 
start  in  a  node  in  a  dfn-interval  i  ;j  end  either  in  j,  or  in  another  node  of  i Notice 
that  dfn(J)  does  not  belong  in  i ;/ 

As  an  example,  consider  Figure  4.1,  which  shows  the  dfn-intervals  of  of  the  pro¬ 
gram  graph  for  the  minmax  program.  These  intervals  are  summarized  in  Table  4.1. 

The  three  intervals  named  i^,  if,,  and  i-j  contain  just  one  basic  block  each.  They 
don’t  abstract  much  interesting  structure,  as  they  coincide  directly  with  one  unit  of 
granularity  of  this  analysis.  On  account  of  this  observation,  it  could  seem  that  we 
could  ignore  all  such  size-1  intervals.  However,  there  is  one  case  where  an  interval 
of  only  one  basic  block  is  interesting:  the  fragment  composed  of  a  block  that  is  a  loop 
ill  itself.  Abstracting  the  loop  into  a  single  fragment  is  a  valuable  operation,  so  we 
cannot  disregard  the  size-1  intervals  completely.  We  arrive  then  at  a  compromise 
that  keeps  the  useful  size-1  fragments  and  discards  those  that  don’t  abstract  any 
interesting  control  structure: 

Depth-first  Interval  Fragments.  Let  i  ;j  be  a  cffii-interval  of  graph  G,  as  above. 
Then,  i  ij  constitutes  a  Depth-First  interval  fragment  of  G  (abbreviated  dfi-fragment) 
if  and  only  if: 

(size(/;y)  >  1)  v  (lesucc(i))  (4.2) 

The  first  term  in  4.2,  (size(/;y)  >  1),  represents  the  non-triviality  of  abstracting  as  a 
unit  some  part  of  the  program  that  contains  more  than  one  fragment.  The  second 
tenn,  (/esucc(/)),  captures  the  non-triviality  of  those  intervals  that  contain  a  single 
fragment  that  happens  to  be  a  loop  in  itself. 

4,2.  Aspects 

The  insight  behind  fragments  is  that  certain  subsets  of  the  entire  program  can  be  con¬ 
sidered  as  indivisible  units  for  purposes  of  analyzing  larger  ones.  This  is  a  fomi  of 
control  abstraction.  The  motivation  for  choosing  one  definition  of  fragment  over  oth¬ 
ers  is  that  certain  control  abstractions  are  especially  useful  towards  the  goal  of  finding 
parallelism. 

For  instance,  dfi-fragments  abstract  some  control  structures  at  the  level  of  basic 
blocks.  We  can  even  give  simple  descriptions  of  each  fragment  in  the  minmax  exam¬ 
ple  (see  Table  4.2).  We  say  that/2  is  the  body  of  the  loop,  because /2  abstracts  the 
structure  that  is  iterated.  In  Table  4.2,  the  intervals  from  Table  4.1  that  are  also  dfi- 
fragments  are  renamed  for  4  to  /^,  to  emphasize  that  only  some  of  the  intervals  are 
also  fragments  under  this  definition.  That  such  characterizations  of  function  are  easy 
to  find  supports  infonnally  the  impression  that  this  particular  set  of  abstractions  can 
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Size 

Name 

Entry;  Exit 

1 

h 

6;7 

4;5 

h 

1;2 

2 

U 

5;7 

h 

3;5 

4 

h 

3;7 

6 

h 

2;8 

7 

io 

1;8 

Table  4.1 .  Intervals  of  the  Minmax  Program 


Size 

Name 

Entry;  Exit 

Description 

Components 

2 

/4 

5;7 

Update  of  Maximum 

/3 

3,5 

Update  of  Minimum 

4 

fl 

3;7 

Body  of  the  Lxjop 

/3-/4 

6 

/l 

2;8 

The  Loop 

h 

7 

/o 

1;8 

The  Program 

f\ 

Table  4.2.  Fragments  of  the  Minmax  Program 


be  useful  to  understand  the  program  starting  from  its  parts.  (The  ultimate  test  of  this 
vague  impression  is,  of  course,  that  the  we  can  write  useful  parallelizations  in  terms 
of  this  analysis.  Chapter  6  presents  my  experiences  in  this  respect,  but  some  informal 
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motivation  here  may  be  useful.) 

As  I  said  before,  considering  all  combinations  of  subsets  of  a  program  is  too 
expensive.  Even  considering  111  combinations  of  fragments  can  be  too  expensive.  I 
need  to  formalize  the  idea  that  only  some  analyses  of  a  fragment  in  its  parts  can  be 
used  for  parallelization,  if  we  want  to  keep  the  running  time  manageable.  My  notion 
of  aspects  of  a  fragment  captures  the  idea  of  restricting  the  parallelization  effort  to 
the  interactions  of  only  some  combinations  of  fragments. 

This  section  presents  the  terminology  needed  for  this  discussion.  I  will  show 
later,  in  Chapter  6,  that  the  restrictions  imposed  by  my  definitions  of  fragment  and 
aspect,  while  prjning  drastically  the  total  amount  of  work  with  respect  to  a  naive  all¬ 
subsets  analysis,  still  manage  to  enable  all  the  useful  transformations  from  the  litera¬ 
ture. 

Table  4.2  introduces  the  notion  of  compositions  of  a  fragment.  Fragment  /4,  for 
instance,  is  a  maximal  subset  of  fragment  /2,  thus  it  is  listed  as  a  component  of  the 
latter.  Although  /4  is  also  a  subset  of  /],  it  is  not  maximal,  and  so  it  is  not  listed. 
Some  definitions  are  needed  here. 

Component.  Fragment  /  is  a  component  of  fragment  g  if  and  only  if  /  is  a 
proper  subset  of  g. 

Maximal  Component .  Fragment  /is  a  maximal  component  of  fragment  g  if  and 
only  if  /is  a  component  of  g,  and /is  not  a  component  of  another  component  of  g. 

Fragment  Composition .  A  composition  of  a  fragment  /  is  a  collection  /] ,  /2, ... 
fic  of  maximal  components  of  /.  Compositions  are  not  necessarily  exhaustive  covers. 

Although  fragments  abstract  control  structure,  they  are  not  totally  devoid  of 
control  structure  themselves.  Composition  formalizes  the  notion  that  further  abstrac¬ 
tion  is  possible,  this  time  at  the  fragment  (instead  of  basic  block)  level.  Such  ab.strac- 
tion  collapses  entire  fragments  down  to  single  nodes. 

Aspect.  Let /be  a  fragment  of  G  and  let  C  =  (/]  /2  •  •  •  fk)  be  a  composition  of 
/.  The  aspect  of  /  under  collapse  of  C,  represented  by_[//C],  is  the  flowgraph  that 
derives  from  that  of  /by  substituting  a  new  single  node  /•  for  all  the  nodes  in  each  /•. 
This  substitution  converts  arcs_entering /■  into  arcs  entering/,  and  arcs  leaving  /  and 
entering/-/  into  arcs  leaving /. 

By  collapsing  each  component  of  a  fragment  to  a  single  node,  we  obtain  alterna¬ 
tive  views  of  the  flow  inside  that  fragment.  Consider  for  instance  Figure  4.2.  There  1 
depict  four  compositions  of  the  fragment /2  (which  is  dfn-interval  3;7). 

I  will  represent  aspects  by  giving  the  fragment  being  considered  and  the  list  of 
the  fragments  being  collapsed.  So,  3;7  considered  without  any  internal  collapse  will 
be  denoted  by  [3;7/()],  while  3;7  with  fragments  3;5  and  5;7  collapsed  to  single  nodes 
will  be  denoted  by  f3;7/(3;5  5;7)]. 

Aspects  can  then  be  used  to  expose  high-level  control  structures.  In  the  view  of 
3;7  provided  by  [3;7/(3;5  5;7)]  we  can  discern  a  very  important  structure:  regardless 
of  the  internals  of  3;5  and  5;7,  fragment  3;7  is  just  a  simple  sequence  of  those  two 
maximal  components.  I  will  use  this  to  obtain  a  concise  encoding  of  parallelization 
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conditions:  I  need  to  provide  only  a  single  rule  for  parallelizing  sequences,  and  will 
depend  on  aspect  formation  to  expose  to  the  same  transformations  sequences  of 
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higher-level  (i.e.,  sequences  composed  of  fragments  more  complex  than  single 
granules). 

4.3.  Aspect  Formation 

The  set  of  aspects  of  a  given  fragment  hierarchy  defines  a  relationship  between  each 
fragment  and  some  sets  of  its  subfragments.  I  use  this  relationship  to  propagate  the 
results  of  the  analysis  upwards  from  simple  fragments  to  their  super-fragments.  The 
character  of  this  relationship  is  that  an  aspect  essentially  records  an  attempt  to  paral¬ 
lelize  a  fragment  by  eliminating  (hopefully)  irrelevant  detail  in  some  of  its  subfrag¬ 
ments. 

The  idea  is  to  take  the  (static)  hierarchy  of  fragments,  and  visit  them  from  small 
to  large,  enumerating  for  every  fragment  all  the  aspects  that  involve  only  fragments 
that  have  been  visited  already.  This  involves  first  selecting  for  each  fragment  which 
subfragments  will  be  used  to  construct  aspects,  and  then  ordering  the  visits  after  a 
topological  sort  of  the  hierarchy,  where  the  partial  ordering  is  fragment  inclusion. 

For  each  fragment  /,  let  Components  \(f)  and  Components 2(f)  stand  for  subsets 
of  the  subfragments  of  /. 

For  each  subfragment  s  in  Components  \  {f),  we  construct  an  aspect  for /by  cal¬ 
culating  the  quotient  [fis].  After  each  such  quotient,  we  consider  the  nodes  that 
remain  uncollapsed  in  the  new  graph  (i.e.,  the  nodes  in  f-s).  We  then  list  the 
members  of  Components 2(f)  that  fit  in  the  remaining  mass.  These  are  the  subfrag¬ 
ments  all  whose  nodes  and  edges  are  members  of  the  nodes  and  edges  of  the  remain¬ 
ing  mass.  Let  5]  be  one  of  them.  Then  we  collapse  5)  in  f-s,  and  repeat  the  pro¬ 
cedure,  starting  now  with  f-s-si.  When  no  more  fragments  fit  in  the  remaining 
mass,  we  have  a  components  list  and  an  aspect. 

I  have  left  unspecified  what  subfragments  of  /are  in  ihe  sets  Components -[if) 
and  Components 2(f)-  This  is  another  design  decision,  exactly  like  the  definitions  of 
granule  and  fragment.  The  choices  range  from  considering  all  the  .subfragments  of/ 
as  Components  to  choosing  only  a  small  number  of  them.  The  first  of  these  sets 
determines  which  subparts  of  a  fragment  stan  the  aspect  formation  process;  the 
second  set  determines  which  subfragments  are  deemed  available  for  further  collapse. 

SPOIL  uses  the  following  strategy  for  aspect  fomiation: 

Components  \  (f)  =  Components 2 CO  =  Maximals if)  (4.3) 

where  Maximalsif)  is  the  set  of  maximal  subfragments  of  /. 

4.4.  Iterators 

We  now  have  a  fragment  hierarchy,  and  a  set  of  aspects  (determined  by  our  choices 
of  Components  \  and  Components 2)-  Now  we  want  to  organize  the  analysis  in  such  a 
way  that  the  eventual  enumeration  order  ("small  ahead  of  large")  be  obtained.  To 
this  end,  rank  all  the  fragments  according  to  the  following  procedure: 

(1)  First,  in  any  order,  all  the  fragments  /  for  which  Suhfragments(f)=0.  There 
have  to  be  some  fragments  that  satisfy  this  condition,  because  there  have  to 
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be  minimal  fragments.  If  /  is  a  subfragment  of  a  fragment  g,  remove  /  from 
the  components  of  g. 

(2)  Repeat  (1)  until  no  unranked  fragments  are  left. 

The  ranking  thus  obtained  represents  a  topological  sort  of  the  fragments,  that 
puts  first  the  fragments  that  have  no  structure,  and  then  successively  larger  fragments, 
in  order  of  their  inclusion  relationship. 

If  we  now  enumerate  the  aspects  of  every  fragment,  in  the  order  of  their  ranking, 
we  will  have  the  guarantee  that  every  fragment  is  enumerated  eventually  after  all  its 
subfragments.  This  is  the  eventual  enumeration  ordering  property. 

I  capture  this  notion  in  SPOIL  by  means  of  iterators .  An  iterator  is  a  function  of 
a  fragment  hierarchy,  a  fragment,  and  a  list  of  fragments,  that  returns  a  list  of  aspects. 
Its  meaning  is  this:  iterator {fhj , history)  =  visits  if  and  only  if  visits  is  the  list  of 
aspects  that  are  needed  to  enumerate  /  completely,  assuming  that  the  nodes  in  history 
have  been  visited  before. 

The  search  for  parallelizations  occurs  in  this  structured  space  of  aspects:  the 
Fragment  Enumerator  of  Figure  1.1  is  essentially  an  iterator  for  the  fragment  that 
comprises  the  entire  program.  For  each  aspect  returned  by  the  iterator,  the 
Transformer  is  asked  to  parallelize  it,  probably  using  the  results  of  having  already 
seen  all  its  subfragments,  according  to  the  eventual  enumeration  ordering. 

4.5.  Consequences  of  the  Bottom-Up  Organization 

Please  consider  again  Figure  4.2.  Fragments  /3=3;5  and  /4=5;7  correspond  to  the 
updating  of  the  running  values  of  the  minimum  and  maximum  found  so  far.  The 
figure  shows  alternative  views  of  the  fragment  /2=3;7.  For  instance,  part  (d)  shows 
that  /2  can  be  read  as  a  simple  sequence  of  nodes.  Fragment  /3  abstracts  basic 
blocks  BB  3  and  BB  4;  while  /4  abstracts  basic  blocks  BB  5  and  BB  6.  Similarly,  the 
communication  patterns  are  also  abstracted:  Arcs  {BB3,BB5)  and  (BB4,BB5) 
merge  into  the  new  arc  (/3,  /4),  representing  the  fact  that  at  this  resolution,  leaving 
/3  implies  entering /4. 

Notice  also  that  the  two  updates  are  “independent”,  in  an  intuitive  sense:  both 
store  new  values  in  different  variables,  and  the  values  stored  by  one  are  never  needed 
by  the  other,  nor  overwrite  values  the  other  may  eventually  use. 

This  suggests  a  possible  rule  of  parallelization,  which  we  can  state  informally  by 
saying  that  given  two  masses  of  code  linked  in  sequence,  they  can  be  executed  in 
parallel  provided  that  (1)  they  don’t  write  on  the  same  locations,  and  (2)  they  don’t 
read  locations  potentially  written  by  the  other.  (This  classical  rule  first  appeared  in 
[Ber66].) 

To  exploit  this  idea,  we  could  look  for  such  a  conspicuous  pattern  as  two  (or 
more)  nodes  chained  together  in  the  shape  of  a  string.  This  plan  raises  two  questions. 
The  first  is  about  the  meaning  of  pattern  recognition  in  the  class  of  program  graphs, 
and  I  address  it  later  (Section  5.2).  The  second  question  is  about  the  global  organiza¬ 
tion  of  the  search.  That  is,  assuming  that  we  have  a  mechanism  to  decide  if  a  given 
graph  matches  a  specified  pattern,  how  do  we  set  up  the  search  in  the  space  of  all  the 
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possible  subgraphs  and  all  the  possible  patterns? 

I  have  already  advanced  my  solution  to  the  second  question.  The  hierarchy  of 
fragments  drives  the  search,  by  presenting  one  fragment  at  a  time  for  consideration 
by  the  transformer.  The  guarantee  of  eventual  enumeration  ordering  provides  the 
necessary  structure  to  the  search:  any  interesting  parallelization  discovered  for  a 
fragment  is  eventually  visible  to — and  can  affect  the  parallelization  of — all  its  super¬ 
fragments. 

Consider,  for  instance,  the  last  time  fi  is  presented  for  transformation.  It  will  be 
true  by  then  that  f-x,  and  have  been  presented  before,  so  the  flow  information  visi¬ 
ble  at  their  boundaries  will  be  known,  thus  enabling  the  evaluation  of  the  conditions 
for  the  independent  sequences  parallelization,  as  outlined  above. 

Similarly,  when  /q  is  presented  as  [1;8/(1;2  2;8)]  we  will  notice  that  it  too  has  a 
string-like  structure,  but  now  we  have  that  2;8  updates  a  variable  defined  by  1  ;2 — 
namely,  the  loop  index.  So,  the  intended  parallelization  won’t  apply  in  this  case. 

4.6.  Fragment  Hierarchy  Interface 

In  order  to  abstract  the  internals  of  the  fragment  enumerator,  I  need  to  assume  some 
interface  for  the  fragment  hierarchy.  I  will  say  that  each  fragment  has  an  associated 
key,  W-s,  fragment  id,  that  can  be  used  to  store  it  in  or  retrieve  it  from  the  fragment 
hierarchy.  This  is  represented  by  the  functions  get_id  (fragment)  and 
get_fragment (id) . 

I  will  also  need  an  iteration  facility  on  fragments.  The  function 
ma}ce__fragment_iterator  (id)  returns  an  object  that  keeps  state  about  an  iteration 
on  all  the  subfragments  of  the  one  designated  by  the  id.  Then,  the  function 
get_next_fraginent  (iterator)  will  return  either  another  subfragment  of  the  one 
on  which  iterator  was  primed,  or  a  distinguished  value  representing  the  exhaustion 
of  the  iterator.  These  iteration  functions  implement  the  eventual  enumeration  order¬ 
ing  guarantee. 

4.7.  Related  Work 

Fragment  hierarchies  are  about  abstracting  detailed  computations.  As  such,  any  pro¬ 
cedural  abstraction  mechanism  could  be  used  to  design  a  fragmeni  decomposition. 

More  concretely  related,  one  can  find  efforts  to  summarize  the  control  structure 
of  a  program.  For  instance,  Lepage  et  al.  [LBR81]  consider  a  multi-level  directed 
graph,  in  which  a  single  node  at  a  given  level  may  stand  for  a  single-entry  subgraph 
of  the  node  immediately  underneath.  Sharir  fSha80]  builds  a  depth-first  search  tree 
starting  from  the  control  flow  graph,  and  then  tries  to  reduce  specific  control  struc¬ 
tures. 

Fragment  hierarchies  are  also  a  matter  of  focus:  the  parallelizer  must  consider 
‘interesting’  pieces  of  its  input.  In  this  sense,  work  that  aims  at  identifying  ‘interest’ 
in  subgraphs  of  a  program  representation  is  related  to  the  work  reported  here.  For 
instance,  consider  the  U-blocks  of  [Ban79].  A  U-block  in  the  dependence  graph  for 
a  program  is  either  a  maximal  cycle  or  an  isolated  point.  Banerjee  tries  to  construct 
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distributed  versions  of  the  code  for  each  U-block,  and  so  Y\-blocks  play  a  role  simi¬ 
lar  to  th?t  of  my  fragments,  in  that  they  are  a  focus  of  analysis.  On  the  other  hand, 
U-blocks  don’t  nest  (as  they  are  maximal  cycles),  and  so  they  can’t  be  used  to  organ¬ 
ize  a  hierarchical  search.  Also,  the  units  of  parallelization  are  then  inside  Y\-blocks, 
thus  being  closer  to  my  notion  of  granule  than  to  my  notion  of  fragment. 

Warren  [War84]  considers  a  dependence-based  program  representation  that 
joins  the  concepts  of  loop  carried  dependence  and  hierarchical  abstraction.  The  idea 
is  to  keep  in  a  single  place  all  the  information  needed  to  reorder  a  loop. 

Lam’s  [Lam88]  VLIW  compiler  for  the  Warp  [AAG87,KuM84]  schedules 
loops  by  means  of  a  hierarchical  reduction  step.  The  intention  is  to  abstract  away  the 
structure  imposed  by  conditionals,  so  that  the  inner  loops  appear  as  single  blocks  to 
the  scheduler  (which  otherwise  would  inhibit  its  search  at  the  conditionals).  This 
hierarchical  reduction  also  drives  the  search:  the  scheduling  terminates  when  the 
entire  program  has  been  reduced  to  a  single  node. 

All  these  investigations  have  abstracted  the  control  structure  of  a  program  in 
order  to  progress  from  small  to  large  pieces.  An  element  that  distinguishes  the 
research  reported  here  is  the  notion  of  aspect,  which  formalizes  the  idea  of  consider¬ 
ing  the  same  piece  of  a  program  under  different  abstractions  of  its  subparts. 


CHAPTER  5 


Exposing  Parallelism:  The  Fragment  Transformer 


The  actual  work  of  parallelism  recognition  occurs  in  the  transformer  of  Figure  1.2. 
This  chapter  describes  how  the  fragment  enumeration  is  eventually  converted  into  an 
execution  set  for  the  program  as  a  whole. 

5.1.  ExecSet  Hierarchy 

Execution  sets  are  defined  by  a  mass  and  a  partial  order.  A  fragmer  ^presents  one 
such  arrangement,  so  each  fragment  has  exactly  one  execution  set.  Thus  we  define  a 
function  execset:  Fragments  ExecSets  to  capture  this  association. 

However,  determining  the  execution  set  for  a  fragment  is  equivalent  to  deter¬ 
mining  precisely  the  dependences  (ergo,  equivalent  to  solving  the  general  paralleliza¬ 
tion  problem).  So  w'e  can’t  (in  general)  obtain  the  precise  execution  set  for  a  frag¬ 
ment.  We  can,  however,  approximate  the  values  of  the  execset  function  with  conser¬ 
vative  execution  sets,  where  by  conser/ative  I  mean  execution  sets  that  are  smaller 
(i.e.,  more  serial)  than  the  ideal  ones. 

Thus  we  do  not  consider  the  execution  set  to  be  a  static  attribute  of  a  fragment. 
Instead,  we  start  wi'n  a  correct  (if,  perhaps,  overly  serial)  execution  set  for  each  frag¬ 
ment,  and  obtain  a  sequence  of  approximations  over  time,  based  or  the  results  of 
work  on  sub-fragments,  until  we  obtain  the  largest  execution  set  thar  can.  How 
large  this  execution  set  is,  in  absolute  terms  and  with  respect  to  the  ideal  one,  is  rela¬ 
tive  to  the  power  of  the  parallelism  detection  mechanism  that  1  describe  in  this 
chapter. 

The  connection  between  fragment  ids  and  execution  sets  also  chan^  :s  with  fhre. 
Transformations  that  add  or  move  code  make  it  undesirable  to  tie  exe  ution  sets  to 
specific  program  graphs.  If  a  transformed  fragment  at  a  given  position  in  me  program 
graph  retains  the  meaning  of  the  overall  program,  1  want  to  consider  its  execution  set 
as  a  description  of  a  valid  alternative  to  the  original  fragment. 

So,  as  a  convention,  I  will  treat  fragment  ids  as  defining  a  position  in  the  pro¬ 
gram,  rather  than  a  specific  program  subgraph.  The  idea  is  the  following:  when  a 
fragment  is  transformed,  the  resulting  execset  is  stored  back  in  the  execution  set 
hierarchy  under  the  same  key  (namely,  the  id  of  the  fragment)  that  the  original 
execset  had.  (Remember  that  the  fragment  hierarchy  and  the  execution  set  hierarchy 
are  isomorphic.)  So  we  have  two  functions  to  access  and  update  current  execset; 

get_cvirrent_execset  (id) 
set_current_execset  (id,  new-execset) 

Aspect  formation  also  causes  execution  sets  to  change  over  time.  As  an  exam¬ 
ple,  consider  the  four  compositions  of  3;7  shown  in  Figure  4.2,  repeated  here  for 
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A  consequence  of  this  evolution  over  time  of  the  execution  sets  is  that  the  typi¬ 
cal  execution  set  expression  is  a  disjunction  of  alternatives.  When  a  parallelization 
provides  a  new  alternative,  it  is  first  checked  for  reducibility  to  one  of  the  already 
known  ones.  This  is  done  by  first  computing  a  reduced  form  for  each  alternative. 
For  instance,  both  (series  A  (series  SC))  and  (series  (series  A  5)  C) 
reduce  to  just  (series  ABC),  which  is  the  form  actually  stored. 

5.2.  Recognition  of  Patterns  in  Graphs 

In  Section  4.5, 1  left  open  the  question  of  w..a'  ,  attem  recognition  in  the  class  of  pro¬ 
gram  graphs  is.  This  question  has  two  aspects:  what  are  we  willing  to  call  a  pattern 
and  how  does  one  decide  if  a  graph  matches  it  or  not. 

These  issues  could  be  resolved  in  their  generality  if  we  had  available  some 
sufficiently  powerful  recognition  technique.  For  instance,  context-free  graph  gram¬ 
mars  would  provide  a  reasonable  setting  for  a  discussion  of  pattern  matching  in  flow 
graphs. 

Regrettably,  the  question  of  the  maximal  class  of  context-free  graph  grammars 
that  can  be  parsed  is  still  essentially  open.  Some  subclasses  have  been  parsed  suc¬ 
cessfully,  so  I  will  argue  for  using  some  parsing  mechanism,  in  as  much  as  the  gram¬ 
mars  thus  parsable  are  sufficiently  expressive. 

SPOIL  implements  a  parser  for  Farrow  et  al.’s  SSFG  (Semi-Structured  Flow 
Graphs)  grammar.  This  is  a  single  grammar,  but  designed  specifically  to  capture  the 
structure  of  flow  graphs.  The  reference  to  this  work  is  [FKZ76],  to  which  the  reader 
is  referred  for  further  details.  See  also  Appendix  C  for  a  quick  summar>'. 

5.3.  The  Catalog  of  Parallelizations 

The  catalog  will  contain  niles  indicating  which  tests  are  needed  to  permit  a  given 
parallelization  and  what  the  results  of  applying  it  are.  In  the  descriptions  that  follow, 
/stands  for  the  fragment  presented  to  the  transformer. 

5.3.1.  Rule  Structure 

Let’s  begin  by  describing  the  structure  of  the  rules.  Rules  will  be  composed  of  a 
fliiarci  and  some  transformations. 

The  guards  are  tests  on  the  properties  of  the  fragment.  They  are  written  in  temis 
of  a  small  number  of  primitive  tests,  organized  recursively.  Guards  have  two  func¬ 
tions.  First  of  all,  they  are  predicates  {does  this  graph  parse  as  a  loop?  do  these  two 
fragments  write  on  the  same  location?)  that  determine  whether  the  rule  applies. 
Second,  they  bind  names  to  objects  discovered  during  the  testing  (for  instance,  the 
parts  of  a  parse,  the  data  flow  sets,  some  relevant  subfragments,  etc.).  Tho.se  names 
are  represented  in  what  follows  as  identifiers  w'hose  first  character  is  a  question  mark. 

The  transformations  consist  of  tw'o  sets  of  effects:  first,  reorganization  and 
compensation,  which  may  create  new  entries  in  the  fragment  hierarchy  and  modify 
pieces  of  code  to  expose  some  parallelism;  and  second,  the  calculation  of  a  new  exe¬ 
cution  set  to  reflect  the  parallelization  accomplished.  This  execution  set  is  reduced 
and,  if  not  otherwise  subsumed  in  pre\'ious  parallelizations,  it  is  included  in  the 
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alternatives  list  for  the  fragment  id  that  is  kept  in  the  ExecSet  hierarchy. 

5.3.2.  Guards 

Guards  are  expressed  in  terms  of  a  set  of  primitive  tests,  connected  by  a  set  of  con¬ 
trolling  connectives.  The  binding  of  variables  is  accomplished  implicitly,  when  a  rule 
takes  a  pattern  as  an  argument,  or  explicitly,  by  means  of  the  match  primitive. 

Although  guards  are  predicates,  and  their  expression  could  then  be  completely 
logical,  I  have  studied  only  guards  defined  procedurally:  guard  satisfaction  is 
described  as  sequential,  recursive,  evaluation  of  each  term  in  the  guard,  following  the 
left  to  right  order  in  each  recursion  level.  This  definition  was  chosen  solely  for 
implementational  convenience.  Considering  a  guard  language  defined  in  purely  logi¬ 
cal  terms  remains  an  area  open  to  investigation. 

The  primitive  tests  used  by  SPOIL  are  described  in  Appendix  A;  this  section 
gives  a  rationale  for  them  and  provides  part  of  the  background  needed  for  understand¬ 
ing  the  example  in  Section  5.4. 

Grammar  tests.  These  consist  of  a  context-free  graph  grammar,  against  which  we 
attempt  to  parse  the  given  fragment.  The  result  of  a  successful  parse  can  be  accessed 
by  means  of  paths  in  the  parse  tree.  For  instance,  define  a  grammar  for  sequences, 
the  S-grammar,  in  this  way: 

Sequence  :  ->  First  ->  Tail  (S-grammar) 

First  -4  :  node  -4 
-^Tail  -4  :  -^Sequence  -4 

I  0 

and  let’s  say  that  fragment /is  derived  by  the  S-grammar  in  the  following  way: 

— >  Sequence  -4  :  -4  First  — >  Tail  -4 

;  -4g  -^Tail  -4 

-^8  -^h^=f 

then  the  pattern  matcher  allows  us  to  refer  to  g  as  f  .Sequence. First,  and  to  h  as 
f  .Sequence. Tail.  (The  syntax  actually  used  by  SPOIL  is  much  less  graceful,  but 
equivalent.) 

Fragment  Structure  tests.  These  are  expressions  of  one  of  the  forms: 

(exists  guard  iter  fragment -id -list  output) 

(forall  guard  iter  fragment -id -list) 

The  meanings  of  these  forms  are  as  follows:  An  exists  fomi  succeeds  if  any  frag¬ 
ment  id  in  the  fragment -id -list  (which  are  always  subfragments  of  /)  satisfies  the 
guard.  These  tests  are  done  by  successively  binding  the  iter  variable  to  the  members 
of  the  list,  and  collecting  the  ones  that  succeed  in  output.  A  forall  form  succeeds  if 


45 


all  the  (current)  subfragments  of  /  in  the  fragment -id -list  satisfy  the  recursive 
guard.  Nothing  is  bound  in  a  forall. 

Data  flow  tests.  These  consist  of  standard  data  flow  equations  (examples  appear  in 
the  rules  given  in  Section  5.3.4).  These  tests  succeed  if  the  equations  can  be  satisfied 
in  terms  of  the  data  flow  information.  These  tests  can  bind  the  results  of  some  set- 
theoretic  manipulations  that  are  needed  to  satisfy  the  equations.  For  instance,  if  an 
equation  requires  that  the  intersection  of  two  sets  of  used  variables  be  non-empty,  we 
can  give  a  name  to  that  set,  so  that  the  actions  can  refer  to  those  variables. 

Execution  Set  tests.  These  tests  compare  known  execution  sets  (obtained  by  means  of 
(get-current-execset  fragment) )  against  each  other  or  against  patterns.  This 
matching  is  a  simple,  syntactic,  unification  on  S-expressions.  At  the  atomic  level,  we 
either  have  the  distinguished  constants  series,  parallel,  sloop,  or  plocp,  or  we 
have  other  execution  sets.  When  we  descend  to  execution  sets  with  no  structure, 
matching  is  by  identity.  The  patterns  include  unification  variables  (denoted  here  with 
names  like  ?List),  these  are  the  names  bound  by  a  successful  guard.  The  tests  are 
expressed  as  calls  to  the  function  match,  which  perfomis  the  actual  unification. 

To  deal  with  choice  branches,  I  use  an  exists-execset  primitive  test.  This 
test  takes  two  arguments;  a  fragment  id  and  a  pattern.  If  the  given  fragment  id’s 
execset  expression  is  a  choice  form,  then  each  of  its  branches  is  tested  against  the 
pattern  (and  bindings  made  if  any  is  successful).  Else,  the  execset  expression  itself  is 
matched  against  the  pattern. 

Connectives.  The  guard  primitives  discussed  above  are  connected  by  the  and,  or, 
and  not  connectives.  These  connectives  are  defined  in  a  procedural,  not  purely  logi¬ 
cal,  way:  their  arguments  are  considered  left-to-right,  and  the  connective  succeeds  or 
fails  accordingly.  Success  is  defined  as  being  able  to  produce  a  set  of  bindings  for  the 
unification  variables;  failure,  as  finding  an  irreconcilable  contradiction.  Each  term  is 
tested  in  an  environment  where  all  the  previous  terms  have  already  left  their  bindings 
available. 

And  succeeds  if  and  only  if  all  of  its  terms  succeed.  The  bindings  produced  by 
each  of  the  terms  are  accumulated,  and  are  available  to  the  rest  of  the  rule.  However, 
as  soon  as  a  term  fails,  the  entire  form  fails  and  no  bindings  are  accumulated.  Or 
succeeds  as  soon  as  a  temi  succeeds.  No  other  temis  are  tried  (which  means  that  the 
rest  of  the  rule  cannot  depend  on  observing  bindings  produced  by  those  terms).  The 
bindings  produced  by  the  term  that  succeeded  are  available,  the  rest  are  abandoned. 
Not  succeeds  if  and  only  if  its  only  term  fails.  No  new  bindings  can  be  generated  by 
a  Not,  not  even  bindings  arising  from  the  partial  success  of  its  argument  (i.e.,  if  the 
argument  succeeds  in  making  some  bindings  and  later  fails,  those  bindings  made 
before  the  failure  are  lost  too). 

5.3.3.  Catalog  Structure 

In  this  work  I  ass"  ne  that  the  catalog  is  flat.  This  is,  it  is  just  a  bag  of  rules.  This 
organization  (or  la  thereof)  makes  it  easy  to  write  rules  without  regard  for  whatever 
else  is  already  in  the  catalog,  but  may  be  inefficient.  For  instance,  many  guards  may 
begin  by  testing  against  the  same  grammar.  The  flat  organization  would  waste  the 


46 


result  of  those  tests,  repeating  them  over  and  over. 

An  organization  that  somehow  indexes  the  rules  by  some  properties  of  the 
guards  is  therefore  recommended  for  actual  implementation,  but  is  not  necessary  for 
this  discussion.  Notice  that  the  relative  simplicity  of  the  guards  makes  it  possible  to 
compare  them  syntactically,  and  thus  facilitates  automatic  indexing.  As  a  simple 
example,  the  user  could  write  the  mles  in  the  flat  model,  and  then  have  the  grammar 
tests  factored  out  automatically,  this  being  trivial  at  least  for  non-recursive  guards. 
This  is  an  economy  afforded  by  expressing  the  guards  in  a  less-than-general 
language. 

The  experience  reported  in  Chapter  6  shows  that  the  flat  organization  does  not 
introduce  unbearable  costs  for  the  test  suite  described  there,  so  no  more  mention  of  it 
is  made  in  this  dissertation,  but  it  remains  an  interesting  area  for  future  research. 

5.3.4.  Rule  Examples 

This  section  shows  a  number  of  examples  of  rules,  written  in  the  language  of  SPOIL’S 
catalog.  Its  purpose  is  to  provide  definiteness  to  the  discussion  above  and  a  point  of 
future  reference  for  the  example  that  follows,  in  Section  5.4.1. 

In  SPOIL,  I  represent  rules  by  S-expressions.  Symbols  of  the  form  ?var 
represent  unification  variables.  Before  each  rule  is  activated,  the  interpreter  will 
assign  the  variables  ?f  and  ?c,  to  the  current  fragment  id  and  the  current  composi¬ 
tion  under  investigation.  Each  rule  is  tested  in  turn:  its  guard  is  evaluated,  and  if  suc¬ 
cessful,  its  transformation  is  run.  The  guard  collects  bindings  for  the  variables  that 
appear  in  the  rules,  and  these  bindings  are  made  available  to  the  transformation. 
Appendix  A  contains  a  list  of  the  operations  that  may  be  used  by  the  transformer. 


47 


Independent  Sequence. 

Guard. 

( :and 

(: aspect  ?f  ?c  ?a) 

(:graph  ?a  ?pg) 

(:parse  ’pg  "sequence"  ?fparse) 

(••match  ?^parse  (:cc  (:cc  ?first  ?tail)  ?exit) ) 

(:part  ?first  ?a  ?f.Seq. First) 

(••part  ?tail  ?a  ?f.Seq.Tail) 

(: flow-attribute  ?f.Seq. First  ()  ruse  ?Use.f.Seq. First) 

(: flow-attribute  ?f.Seq.Tail  ()  :use  ?Use.f .Seq.Tail) 

(: flow-attribute  ?f.Seq. First  ()  :def  ?Def.f.Seq. First) 
(.■flow-attribute  ?f. Seq.Tail  ()  :def  ?Def.f .Seq.Tail) 

(: intersect ion  ?Def.f.Seq. First  ?Use.f .Seq.Tail  ?1) 

(: intersection  ?Use.f.Seq. First  ?Def.f .Seq.Tail  ?2) 

(: intersection  ?Def.f.Seq. First  ?Def.f .Seq.Tail  ?3) 

(:enptyset  ?1)  ;not  flow  dependent 

(:enptyset  ?2)  /not  anti  dependent 

(:enptyset  ?3)  /not  output  dependent 

) 

Transformations. 

(:  apply 

( :add-to-execset  ?f 
( :parallel 

( : execset  ?f . Seq . First) 

(:execset  ?f .Seq.Tail) )) ) 

This  rule  uses  Bernstein’s  conditions  to  determine  when  two  parts  of  a  program  are 
independent. 

First  we  calculate  the  aspect  of  ?f  under  collapse  of  the  fragments  whose  ids, 
are  in  ?c.  Then  we  extract  the  graph  of  the  fragment  ?f,  with  the  components 
already  collapsed.  That  graph  is  then  parsed  against  the  SSFG  grammar  (the  string 
"sequence"  that  appears  in  the  parse  primitive  is  just  a  comment’).  Then  we  compare 
the  parse  with  a  pattern  that  represents  two  nodes  in  a  row.  The  pattern  can  be  read 
this  way:  A  computation  followed  by  a  computation  (:cc),  where  the  first  computa¬ 
tion  is  another  computation  followed  by  a  computation,  whose  first  part  is  named 
?first  and  whose  second  part  is  named  ?tail.  The  second  part  of  the  outermost 

'  These  comments  arc  a  historical  artifact.  An  early  version  of  SPOIL  tried  to  deal  with  more  than  one 
grammar  at  a  time,  the  slot  currently  taken  by  the  comment  w'as  then  reserved  for  a  grammar 
reference.  After  that  slot  became  useless  (because  sroiL  uses  the  SSFG  grammar  only),  I  left  it  in  use 
because  the  comment  helps  to  understand  the  rule. 
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:cc  is  named  ?exit. 

Then  the  :part  query  tries  to  determine  if  the  subgraphs  ?first  and  ?tail 
so  determined  correspond  to  fragments.  If  that  is  the  case,  their  flow  attributes  of 
usage  and  definition  are  computed,  and  three  intersections  taken;  ?1  represents  the 
definitions  in  First  that  are  used  in  Tail,  ?2  represents  the  definitions  in  Tail  that  are 
used  in  First,  ?3  represents  the  definitions  common  to  First  and  Tail.  The  guard  is 
satisfied  if  the  three  intersections  are  empty. 

When  that  happens,  the  transformer  applies  the  transform.  It  just  upgrades  the 
execution  set  of  the  given  fragment  with  the  option  of  running  the  two  parts 
discovered  during  the  parse  in  parallel. 

Forced  Series. 

Guard. 

( :and 

(: aspect  ?f  ?c  ?a) 

(:  graph  ?a  ?pg) 

( rparse  ?pg  "sequence"  ?fparse) 

(rmatch  ?fparse  (:cc  (:cc  ?first  ?tail)  ?exit) ) 

(:part  ?first  ?a  ?f.Seq. First) 

(:part  ?tail  ?a  ?f.Seq.Tail) 

( : f low^attribute  ?f.Seq. First  ()  ruse  ?Use.f.Seq. First) 

( : f low^attribute  ?f.Seq.Tail  ()  ruse  ?Use.f .Seq.Tail) 

( r flow-attribute  ?f.Seq. First  ()  rdef  ?Def.f.Seq. First) 

( r flow-attribute  ?f. Seq.Tail  ()  'def  ?Def.f .Seq.Tail) 

( r intersection  ?Def.f.Seq. First  ?Use . f . Seq . Tail  ?1) 
(•.intersection  ?Use.f. Seq. First  ?Def . f . Seq . Tail  ?2) 

( r intersection  ?Def . f . Seq . First  ?Def . f . Seq . Tail  ?3 ) 

(ror 

(rnot  ( r enptyset  ?1) )  /flow  dependent 
(rnot  (rtjrptyset  ?2))  /anti  dependent 
(rnot  ( ; enptyset  ?3) )  /output  dependent 
)) 

Transformations. 

(raiply 

( r  add-to-execset  ?f 
( r  series 

( r  e>;  '’cset  ?  f .  Seq .  First ) 

(rexecset  ?f  .SJeq.Tail) ) ) ) 

The  guard  in  this  rule  is  essentially  the  negation  of  the  previous  one:  instead  of 
demanding  that  all  the  intersections  be  empty,  this  guard  demands  that  at  least  one  he 
not  empty. 

The  transfonnation  is  also  simple:  it  just  adds  to  the  execution  set  for  ?f  the 
requirement  of  executing  both  parts  strictly  sequentially.  (But.  of  course,  later 
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reorganizations  and  compensations  could  enable  a  parallelization  that  overrode  this.) 

Loop  Recognition. 

Guard. 

( rand 

(: aspect  ?f  ?c  ?a) 

(:  graph  ?a  ^fpg) 

(rparse  ?fpg  "loop"  ?fparse) 

( ror 

(rmatch  ?fparse  (;cc  (rloqp  (red  ?Body  ?Test) )  ?Exit) ) 
(rmatch  ?fparse  (:cc  (rloqp  (rdc  ?Test  ?Body) )  ?Exit) ) ) 

(rpart  ?Body  ?a  ?LoopBody) ) 

Transformations. 

(r  apply 

( r add-to-execset  ?f 
( r  sloop  ( r  execset  ?LoopBody) ) ) ) 

This  rule  locates  the  loops.  It  looks  for  parses  that  indicate  a  loop  with  the  ?Test 
either  at  the  top  or  at  the  bottom.  The  SSFG  grammar  cannot  generate  arbitrary 
loops,  but  can  handle  large  classes  of  almost-structured  code,  especially  loops  with 
multiple  exits.  So  it  could  be  used  to  support  more  general  rules  (with  more  branches 
in  the  or  that  inspects  the  results  of  the  parse).  The  ones  used  above  are  sufficient  for 
most  purposes  in  languages  like  C,  where  loops  are  rarely  implemented  as  nests  of 
if  s  and  gotos,  but  rather  come  from  structured  constructs. 

Loop  Distribution. 

Guard. 

( :and 

(: execset  ?f  ?e) 

(rmatch  ?e  (: sloop  (: parallel  .  ?parts)))) 

Transformations. 

(:  apply 

(:map  ( :new-execset  (:  sloop  Tpa'i:)  ?newloqp) 

?part  ?parts 
?newloop  ?newloops) 

( rccmbine-execsets  rparallel  ?newloqps  ?neweset) 

(: add-to-execset  ?f  ?neweset) ) 

This  rule  can  be  explained  as  the  following  procedure; 

(1 )  Determine  if  the  current  execution  set  is  a  serial  loop  of  parallel  parts. 

(2)  Iterate  on  the  parts.  For  each  of  them,  create  a  new  execution  set,  a  simple 
serial  loop  of  that  part.  Combine  all  these  e.xecution  sets  into  a  parallel  of  all 
of  them,  then  provide  that  new  execution  set  as  a  choice  for  the  execution  set 
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of?/. 

5.4.  Example 

Now  we  go  through  the  parallelization  of  the  mixmax  example  step  by  step.  The  flow 
graph  and  fragment  structure  of  this  program  are  repeated  here  for  convenience. 
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We  are  going  to  carry  out  this  example  under  the  assumption  that  we  are  using  a 
catalog  that  contains  exactly  the  rules  given  as  examples  in  Section  5.3.4. 

The  order  of  visit  is  calculated  by  obtaining  the  largest  components  of  each  frag¬ 
ment  and  computing  the  difference  of  the  masses  of  the  given  fragment  and  each 
maximal  component.  Then  we  try  to  fit  the  masses  of  other  subfragments  in  the  resi¬ 
dual. 

One  such  order  of  visit  is  given  below.  The  asterisks  on  the  left  indicate  those 
fragments  that  are  going  to  be  discussed  more  carefully  below. 

[i;8/0] 

*  [2;8/0] 

[3;7/0] 

[3;5/0] 

[3;7/(3:5  6;7)] 

[-‘5;7/()] 

*  [3;7/(3;5  5;7)] 

*  [2;8/(3;7)] 

[i;2/0] 

[l;8/(3;7)] 

5.4.1.  Move-by-Move  Run 

At  the  beginning  of  the  transfomiation  process,  a  new  execset  hierarchy  is  con¬ 
structed.  It  is  initialized  trivially,  so  that  fragment  fi  has  as  initial  execution  set 
(execset/,) .  Then  each  of  the  compositions  presented  above  is  analyzed  for  possi¬ 
ble  parallelization.  As  we  progress,  some  of  the  compositions  match  pan  of  some  of 
the  guards,  but  fail  on  others.  For  instance: 

[2;8/()J  _ 

This  fragment  corresponds  to  the  Iriop  of  ;  t'  'e  Of/;an,  con  cied  v.  iih 

no  simplification  of  its  intcma!  .mc'erc  A't.:  :.'ugi/  i;6  is  clc  irly  a  ;• 

to  satisfy  the  loop  re-  ognition  guards,  because  its  parse  fails  to  have  exactly  one 
node  standing  for  th  .  bod^  of  the  loop.  The  conditions,  as  seen  above  in  the 
Loop  Recognition  rule  are: 

(:or  (:match  ?fjarse  (:cc  (:loop  (:cd  ?Body  ?Test))  ?Exit) ) 

(imatch  ?f^rse  (:cc  (:loop  (:dc  ?Test  ?Body) )  ?Exit) ) ) 

and,  as  can  be  seen,  there  is  no  binding  for  ?Body  when  we  consider  the  loop 
2;8  with  all  its  structure  visible. 

For  various  reasons,  none  of  the  interv’ening  aspects  completely  satisfies  the  guard  of 
any  rule  until  we  get  to: 

[3;7/(3;5  5;7)] 

Here  we  find  that  the  fragment  in  question  (also  named  fi,  refer  to  Table  4.2) 
satisfies  the  S-grammar  test.  In  addition,  we  can  identify  two  sub- fragments 
(3:5=/3  and  5;7=/4)  that  are  independent. 
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The  grammar  tests  binds  ?f.seq.first  to  f-^,  and  ?f.seq.tail  to  f^.  With  these 
bindings,  the  action  for  this  rule  constructs  a  new  alternative  reading: 

(parallel  (execset/3) 

(execset /4) ) 

From  this  alternative  and  the  original  trivial  execset,  the  transformer  will  con¬ 
sider  this  new  execset: 

(choice 

(pacrallel  (execset/3) 

(execset  f^)) 

(execset  /2 ) ) 

The  reducer  knows  it  is  trying  to  provide  a  new  execset  for  /2,  so  upon  recog¬ 
nizing  i;,  as  a  subform,  it  simplifies  the  execset  to  just  the  (parallel  •  •  •  )  ver¬ 
sion. 

[2;8/(3;7)] 

This  fragment  begins  with  the  execset  (execset  /] ) .  It  matches  the  loop 
recognition  rule,  which  provides  a  new  reading  as  a  serial  loop.  That  reading 
changes  the  execset  of /i  to 

(sloop  (execset fi)) 

Now,  the  same  fragment  matches  the  loop  distribution  rule.  The  ?Parts  are 
recognized  to  be  and  f^.  With  this  information,  a  new  execset  is  built: 

(parallel  (sloop  (execset/3)) 

(sloop  (execset /4 ) ) ) 

The  choice  between  these  execsets  cannot  be  simplified  by  the  reducer.  The 
fomier  execset  involves  running  the  main  loop  serially,  but  using  two  parallel 
procr'^es  pe:  iteiaiion;  while  the  new  one  implies  a  parallel  of  two  sequential 
loops.  Tl’.ese  alternatives  don’t  reduce  to  each  other  syntactically,  as  per  the 
execset  identities  in  §3.4,  nor  does  one  of  them  subsume  the  other  uncondition¬ 
ally.  In  consequence, /i  acquires  the  execset: 

(choice 

(parallel  (sloop  (execset/3)) 

(sloop  (execset f 4))) 

(sloop  (execset /2 )) ) 

Please  notice  that  the  relative  simplicity  of  the  rules  does  not  prevent  them  to 
match  this  aspect  of  the  loop  (in  contrast  to  their  failure  with  respect  to  [2;8/()]).  The 
economy  afforded  by  aspect  formation  is  seen  again  in  this  simplicity  of  the  loop 
rules,  that  are  spared  the  need  to  match  the  unbounded  variety  of  structure  of  general 
loop  bodies. 

When  the  last  fragment  has  been  parallelized,  we  have  an  execset  hierarchy  that 
assigns  trivial  execsets  to  all  the  fragments  but  for  /o,  /],  and  fi-  Replacing  these 
execsets  recursively,  the  execution  set  for  the  loop  appears  as: 
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(choice 

(parallel 

(sloop 

(execset/3)) 

(sloop 

(execset/4) ) ) 

(sloop 
(parallel 
(execset  /3) 

(execset  /4)))) 

which  expresses  the  sources  of  parallelism  encoded  ;n  the  little  catalog  that  are  also 
present  in  the  program. 

5.5.  Execution  Set  Hierarchies 

The  final  data  structure  relevant  to  the  Transformer  is  the  Execution  Set  Hierarchy. 
This  structure  keeps  track  of  the  association  between  fragment  ids  and  their  current 
(time-evolving)  execution  set  expressions. 

Initially,  the  association  is  trivial:  with  fragment  /,  we  associate  the  execution 
set  (execset  f) .  The  Transformer  modifies  this  association  as  explained  above. 
Therefore,  this  staicture  eventually  provides,  for  every  fragment,  its  definitive  execu¬ 
tion  set. 

However,  reaching  that  final  execution  set  requires  a  cenain  attention  to  detail 
that  complicates  the  intermediate  stages.  This  section  is  devoted  to  explaining  those 
details. 

5.5.1.  Implicit  and  expanded  execution  sets 

Consider  three  fragments  /,  g,  and  It,  progressively  larger.  At  a  certain  stage  in  the 
transfomiation,  the  current  value  of  their  execution  sets  may  be  a  function  of  their 
immediate  descendants,  for  instance: 

(execset  h)  =  (choice  . . .  (execset  g)  . . . ) 

(execset  g)  =  (sloop  . . .  (execset  f)  . . . ) 

Obviously,  the  execution  set  of  h  admits  a  different  expression,  namely 
(choice  ...(sloop  ...  (execset  f)  ...)  ...).  This  expresses  the  execu¬ 
tion  set  of  h  in  ultimate  terms:  all  the  appearances  of  the  form  (execset  x)  refer  to 
fragments  x  that  cannot  be  expressed  in  any  other  way. 

Given  a  fragment  hierarchy  F,  its  associated  execution  set  hierarchy  E,  and  a 
fragment  /  in  F,  1  say  that  the  Expanded  Execset  Expression  for  /  is  the  execset 
expression  that  does  not  contain  any  (execset  jc)  form  for  any  fragment  x  that  has  a 
non-trivial  execution  set  in  E. 

The  expanded  form  o^  an  execset  expression  relative  to  the  hierarchies  E  and  E 
can  be  obtained  by  recursively  replacing  the  references  to  trivial  execset  expressions 
with  their  (then  current)  non-trivial  counterparts. 
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An  execset  that  isn’t  expanded  with  respect  to  the  relevant  hierarchies  is  said  to 
be  in  an  implicit  form. 

The  possible  difference  between  the  implicit  and  the  expanded  forms  is  another 
redundancy  in  execution  set  representation,  as  I  commented  in  Section  3. 4. 1.5.  This 
redundancy  is  different,  however,  in  that  it  can  be  shown  only  relative  to  current  exe¬ 
cution  set  and  fragment  hierarchies.  In  general,  showing  the  equivalence  of  the 
expanded  and  implicit  forms  doesn’t  follow  from  the  syntactic  structure  of  the 
execsets  involved  alone,  but  depends  on  the  structure  of  the  program  and  on  the  his¬ 
tory  of  previous  successful  parallelizations. 

5.5.2.  Propagating  changes  to  execution  sets 

In  general,  the  Transformer  produces  implicit  execution  sets.  This  can  be  seen  by 
observing  that  the  niles  for  adding  new  execution  sets  use  as  pieces  the  components 
of  the  current  aspect.  Those  components  are  not  required  to  have  trivial  execsets. 
However,  matching  execution  sets  that  remain  implicit  is  difficult.  So  there  is  incen¬ 
tive  for  keeping  both  the  implicit  and  the  expanded  forms.  As  a  matter  of  implemen¬ 
tation  strategy,  then,  we  keep  both  forms  at  all  times.  At  the  end  of  the  paralleliza¬ 
tion,  we  need  only  the  expanded  one.  So,  when  do  we  need  the  implicit  forms? 

Well,  we  need  the  implicit  forms  only  in  the  intermediate  stages.  They  keep  the 
connection  between  fragments  and  their  subfragments  that  have  triggered  paralleliza¬ 
tion.  In  the  example  above,  the  form  (choice  ...  (execset  g)  ...) 
remembers  that  g  and  h  are  related.  If  we  replace  it  immediately  by  the  expanded 
form,  we  would  run  the  risk  of  losing  later  opponunities  for  parallelization. 

This  would  happen  as  follows:  The  eventual  enumeration  order  guarantees  that 
the  Transfomier  will  visit  large  after  small,  but  nothing  more.  So  it  is  perfectly  possi¬ 
ble  for  us  to  visit  /,  then  g,  when  parallelizing  h.  If  we  replace  at  the  first  success  the 
execution  set  for  h  with  its  expanded  form  (in  temis  of  /),  all  traces  of  the  connection 
with  g  will  be  lost.  If  we  later  succeed  in  parallelizing  g,  the  eventual  enumeration 
order  guarantees  that  we  still  have  a  chanc*  to  look  at  h  again.  Regrettably,  that  new 
attempt  couldn’t  benefit  from  the  intermediate  parallelization,  because  the  appearance 
of  g  in  the  execset  expression  for  li  has  been  lost. 

This  is  the  reason  for  keeping  both  forms  of  the  execution  set  at  all  times.  The 
implicit  version  remembers  the  connection  between  a  fragment  and  the  aspects  under 
which  it  was  parallelized;  the  expanded  version  shows  the  ultimate  effect  of  all  the 
parallelizations.  To  compare  execsets,  the  guard  evaluator  refers  to  their  expanded 
forms;  to  update  execsets,  the  transform  evaluator  modifies  first  "he  implicit  form,  but 
this  modification  isn’t  enough  to  maintain  the  correctness  of  the  hierarchy,  so  the 
transfomi  evaluator  also  updates  the  expanded  fomi  of  the  fragment  just  parallelized, 
and  then  re-expands  the  implicit  forms  of  all  the  ancestors  of  that  fragment,  starting 
by  the  nearest  ancestor  and  going  upwards  in  the  hierarchy.  Updating  the  execset 
expressions  in  this  order  is  necessary  to  guarantee  that  the  effect  of  the  transforma¬ 
tion  propagates  correctly  to  all  the  fragments  that  could  be  still  parallelized  by  using 
this  information. 
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To  summarize,  we  start  with  trivial  execution  sets  as  both  the  implicit  and  the 
expanded  forms  of  every  fragment.  When  a  transformation  occurs,  the  implicit 
execset  is  the  one  directly  returned  by  the  Transformer.  That  execset  expresses  the 
transformation  in  temis  of  the  aspect  that  was  involved.  We  add  this  execution  set  to 
the  choices  kept  in  the  implicit  form,  and  reduce  it.  Then  we  expand  the  reduced 
form  with  respect  to  the  current  hierarchies,  and  remember  that  expanded  form  along 
with  the  implicit  one.  Finally,  we  climb  the  hierarchies,  re-expanding  the  current 
implicit  forms  of  all  the  ancestors  of  the  modified  fragment. 

5.6.  Discussion 

A  few  observations  are  still  desirable.  First  of  all,  we  can  judge  the  complexity  of  a 
parallelizer  by  inspecting  the  computational  power  of  the  rule  set  used. 

The  reason  for  specifying  the  primitive  guard  predicates,  and  for  limiting  the 
guards  to  using  them,  is  precisely  to  have  a  chance  to  observe  the  complexity  of  test¬ 
ing  for  parallelism  in  a  single  place,  rather  than  spread  all  over  the  compiler.  The 
rule  set  discussed  depends  essentially  on  linear  unification,  set  operations,  and  graph 
parsing.  Arithmetic  and  other  universal  modes  of  computation  were  intentionally  not 
used,  to  avoid  adding  power  that  wasn’t  absolutely  needed.  Of  course,  set  operations 
can  be  used  to  bring  in  arithmetic,  but  the  niles  would  become  impossibly  long  then. 
The  interest  was  in  minimizing  the  temptation  to  implement  the  parallelizations  via 
arbitrary  code  within  the  rules  (then  a  single  rule  could  conceivably  do  all  the  work!). 

The  conclusion  of  this  study  was  to  confirm  the  impression  that  the  complexity 
of  parallelization  can  be  reduced  to  the  complexities  of  data  and  control  flow 
analysis,  and  of  graph  ptirsing,  glued  together  by  unification  of  S-expressior.s.  It 
remains  an  open  question  to  characterize  the  minimal  power  needed  by  those  com¬ 
ponents.  Chapter  6  shows  a  possible  arrangement  with  respect  to  graph  parsing,  but 
no  claim  is  advanced  as  to  its  minimality. 

Another  formalization  of  interest  is  that  of  relative  completeness  of  a  parallel¬ 
izer.  The  general  case  of  parallelization  is  undecidable,  on  account  of  the  undecida¬ 
bility  of  certain  data  flow  problems  [Ber66],  so  we  cannot  expect  to  obtain  an  algo¬ 
rithm  that  finds  all  the  conceivable  parallelism  in  arbitrarj'  ordinary  programs.  How¬ 
ever,  the  exhaustive  search  procedure  used  in  my  Transfonner  affords  me  a  relative 
sense  of  completeness:  my  Transformer  finds  any  parallelization  that  has  been 
expressed  in  its  rule  catalog  and  whose  guard  can  match  in  the  program  under  con¬ 
sideration,  perhaps  after  other  transformations  have  exposed  a  structure  of  interest. 

To  see  why,  observe  that  the  eventual  enumeration  order  ensures  that  syn¬ 
thesized  parallelizations  (i.e.,  those  made  possible  only  after  recursive  consideration 
of  the  sub-fragments)  are  successful  when  at  all  possible.  As  for  inherited  paralleli¬ 
zations  (i.e.,  parallelizations  made  possible  by  inspecting  a  super-fragment),  I  claim 
that  the  hierarchical  nature  of  the  fragment  structure  ensures  at  least  that  one  can 
write  the  rule  to  match  the  structure  of  interest  in  the  context  of  the  larger  fragment. 

These  issues  deserve  a  more  detailed  formal  analysis.  The  ideal  setting  for  that 
analysis  is  in  general  program  transformation  theory,  and  so  it  may  pay  to  embed  the 
question  of  relative  completeness  (of  a  Transfonner  with  respect  to  a  Catalog)  in  the 
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question  of  relative  completeness  of  an  arbitrary  program  transformation  scheme  with 
respect  to  non-deterministic  rewriting  systems.  (The  non-determinism  may  be 
needed  to  account  for  having  to  choose  which  aspect  actually  exposes  the  structure 
that  permits  parallelization.)  Accomplishing  this  would  afford  us  a  declarative  paral¬ 
lel  izer. 

5.7.  Related  Work 

The  idea  of  collecting  transformations  that  could  improve  the  performance  of  gen¬ 
erated  code  seems  to  have  started  with  Allen  and  Cocke’s  catalog  [A1C72]. 

Parafrase  isolated  transformations  in  specific  modules,  and  insisted  on  a  source- 
to-source  mode  of  operation.  Thereby,  one  could  experiment  with  reordering  (or 
repeating)  transformations.  Each  of  the  independent  modules  could  be  considered  a 
member  of  a  loose  catalog  of  general  transformations. 

Aiken  [Aik88]  shows  a  parallelizer  that  uses  a  small  set  of  rules  (four)  to  accom¬ 
plish  its  goals.  His  catalog  is  tightly  woven  with  the  control  structure  of  the  com¬ 
piler. 

I  want  to  point  out  that  the  literature  often  lumps  together  transformations  of 
dissimilar  natures.  I  will  argue  here  that  the  clear  distinctions  introduced  by  my 
model  clarify  the  proper  role  of  those  transformations. 

For  instance,  consider  the  transformations  discussed  by  Polychronopoulos  in 
[P0I88].  He  distinguishes  some  common  optimizations,  like  induction  variable  sub¬ 
stitution  and  loop  unrolling,  that  apply  generally  to  all  optimizers.  These  transforma¬ 
tions  do  not  necessarily  exnose  parallelism  by  themselves,  but  prepare  the  program 
for  future  transformation,  fhen  he  discusses  (Section  2.3)  transformations  that  are 
ostensibly  specific  to  vectorization/parallelization. 

Among  those  transfomiations,  he  discusses  loop  interchange  (Section  2.3.2), 
loop  vectorization  (Section  2.3.1),  and  loop  blocking  (Section  2.3.5).  Observe  that 
the  first  one  converts  the  sequential  program  into  another  sequential  program  (thus 
belonging  properly  into  the  front  end  of  my  design);  the  second  is  a  real  paralleliza¬ 
tion,  and  of  rights  belongs  in  the  catalog  of  my  model;  and  ;he  third  one  actually 
seems  to  hide  some  parallelism! 

To  see  why,  let's  review  what  loop  blocking  is.  It  reduces  to  converting  a  sin¬ 
gle  loop  into  two  nested  loops,  such  that  the  inner  loop  proceeds  in  chunks  of  uniform 
size.  For  instance,  adapting  the  example  in  [P0I88,  p.  26],  we  get  from: 

for  (i  =  0;  i  <  N;  i-H-) 
a[i]  =  b[i]  -t-  c[i]; 

the  new  program: 

for  (j  =  0;  j  <  N;  j  4=  K) 

for  (i  =  j;  j  <  MIN(j-4K,  N),  i-4-4) 
a[i]  =  b[i]  +  c[i]; 

The  motivation  for  this  transformation  is,  presumably,  to  adapt  to  architectural 
characteristics  (like  the  number  of  vector  registers).  Both  of  the  loops  above  have 
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essentially  the  same  execution  set:  (ploop  (execset  f) ) ,  where /is  the  name  of 
the  fragment  that  contains  exactly  the  assignment  statement. 

Once  one  adopts  the  model  discussed  in  this  work,  the  role  of  loop  blocking  is 
clear:  it  is  actually  a  back  end  transformation,  most  certainly  not  necessary  in  the 
parallelization  catalog. 

We  will  see  later  that  SPOIL’S  catalog  can  represent  front  end  and  back  end 
transformations,  in  addition  to  the  transformations  that  properly  belong  there  (i.e., 
those  transformations  that  enlarge  execution  sets).  This  flexibility  is  useful  to  the 
implementor  of  specific  parallelizers,  but  there  is  now  a  characterization  that  will 
recommend  where  to  put  specific  transformations.  One  can  use  that  characterization 
as  a  criterion  to  decide  between  different  configurations. 


CHAPTER  6 


SPOIL:  Practical  Experience 


This  chapter  presents  the  experience  gained  through  SPOIL.  First  I  describe  the 
implementation.  Then  I  consider  the  observed  costs  of  various  design  decisions. 
These  costs  can  be  used  to  balance  the  choices  available  to  the  designer,  namely  the 
grain  of  the  analysis  and  the  thoroughness  of  the  search  among  the  possible  transfor¬ 
mations,  versus  the  running  time  of  the  compilation.  Finally,  1  discuss  the  coverage 
of  the  transformation  space  explored  by  the  catalogs  used  in  these  experiments. 

The  experiments  described  here  have  several  purposes.  The  first  is  to  validate 
the  attempt  to  separate  parallelism  detection  from  the  other  chores  of  the  parallelizer. 
A  second  purpose  is  to  test  the  specific  combination  of  grain  and  composition  stra¬ 
tegy  used  in  SPOIL’S  fragment  hierarchy.  A  third  objective  is  to  collect  and  validate  a 
catalog  of  modest  size,  yet  capable  of  encoding  useful  transformations. 

6.1.  The  Input  to  SPOIL 

This  section  describes  the  language  accepted  by  SPOIL,  and  discusses  the  conse¬ 
quences  of  some  of  the  decisions  that  derive  from  that  choice  of  input. 

SPOIL  processes  a  subset  of  C  fKeR89].  This  subset  excludes  the  notion  of  vola¬ 
tility,  and  is  restricted  to  one  function  ner  unit  of  compilation  (i.e.,  per  .c  file)'. 

The  choice  of  a  subset  of  C  w.i.v  suggested  by  a  number  of  technical  conveni¬ 
ences,  not  the  least  being  the  availability  of  the  GNU  C  compiler  [Sta89].  It  provides 
convenient  access  to  intermediate  representations,  which  I  used  to  generate  the  input 
expected  by  SPOIL,  as  described  below  in  Section  6.2. 

C  is  widely  used,  and  so  should  be  an  interesting  target  for  parallelization. 
However,  it  has  not  received  attention  proportional  to  its  popularity,  probably 
because  it  has  been  mainly  used  as  a  systems  programming  language,  an  area  of 
application  not  traditionally  concerned  with  parallel  speedup.  As  C  spreads  its 
domain  to  application  programming  one  can  expect  that  interest  in  its  parallelization 
will  increase. 

One  previous  parallelizer  for  C  is  due  to  Allen  and  Johnson  [A1J88],  who  have 
worked  on  compiling  C  for  a  multiple  SIMD  machine  (Ardent’s  Titan).  Their  insight 
was  to  raise  the  semantic  level  of  their  intermediate  representation,  in  order  to  make 


'  These  limitalJons  are  not  inherent  in  the  general  model  discussed  in  this  dissertation,  but  are 
conveniences  to  the  prototype  implcmcnuition.  When  discussing  SPOIL  it  is  important  to  distinguish 
those  features,  advantages  or  disadvantages,  that  arc  induced  by  the  theory  discussed  here,  from  those 
other  features  that  are  ju.st  accidents  of  the  prototype  implementation. 
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the  dependences  stand  out.  The  abstraction  provided  by  my  fragment  hierarchy  has  a 
similar  purpose,  but  differs  in  that  it  imposes  no  details  on  the  actual  intermediate 
representation  used,  while  Allen  and  Johnson  describe  a  specific  intermediate 
representation. 

SPOIL  relies  on  a  front  end  to  parse  and  check  the  semantics  of  the  input  pro¬ 
gram,  and  accepts  as  its  input  proper  an  intermediate  representation  of  the  program, 
where  all  syntactic  characteristics  of  C  have  been  eliminated. 

6.2.  Implementation  Considerations 

SPOIL  is  written  in  Common  Lisp  [Ste84].  It  was  developed  and  tested  using  KCl 
(Kyoto  Common  Lisp,  [YuH85])  and  KCl’s  AKCL  port. 

SPOIL  uses  GCC  as  heavily  as  possible  to  avoid  having  to  deal  with  problems 
solved  by  existing  technology,  namely:  lexical  and  syntactic  analysis,  type  checking, 
and  conversion  into  a  register-transfer-level  (RTL)  representation  [DaF84J.  SPOIL 
reads  in  the  RTL  representation  of  the  program,  collects  the  instructions  into 
granules,  and  computes  granule-level  data  flow  information. 

The  proof-of-concept  nature  of  SPOIL  dictated  a  number  of  restrictions  on  the 
fomiat  of  the  programs  used  for  the  experiments,  the  most  important  being  the  limita¬ 
tion  to  one  function  per  compilation.  Interprocedural  analysis,  though  an  important 
issue  in  itself  and  in  its  possible  applications  to  parallelization,  would  not  add  much 
weight  to  the  test  of  the  ideas  exposed  in  this  dissertation,  so  no  effort  was  made  to 
parallelize  across  procedure  calls.  To  accommodate  interprocedural  analysis  in  my 
model,  one  needs  to  define  fragment  hierarchies  that  span  the  call  graph,  and  extend 
the  dataflow  information  across  procedure  boundaries  (see 
lCal88,  C0K88,  HRB88,  Li88]  for  examples  of  the  analyses  that  would  be  needed  to 
extend  my  model). 


Figure  6. 1 .  From  C  to  SRTL 
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Figure  6.1  shows  schematically  the  way  a  C  program  is  converted  into  the  input 
for  SPOIL.  Given  a  C  function  in  a  file  prog.c,  we  run  GCC  on  that  file,  producing 
files  prog.c. flow,  prog.c.greg,  and  prog.s.  The  first  two  contain  alternative 
views  of  an  intermediate  representation  for  the  program,  the  last  one  contains  assem¬ 
bly  code.  The  intermediate  representation  (RTL)  is  described  in  [Sta89].  (See  also 
[DaF84].) 

The  RTL  output  is  not  directly  readable  by  SPOIL.  The  program  rtl2srtl, 
reproduced  below  in  Figure  6.2,  cleans  it  up,  producing  a  file  prog.srtl,  which  is 
suitable  as  input  for  SPOIL. 


# !  A>in/sh 

echo  "(  ;;  RTX  list" 

#  Eliminate  header, 
sed  \ 

-e  '/'[“(]/d'  \ 

-e  '/'(/{ 

:loop 

n;  b  loop 

}' 


#  Eliminate  flags  and  machine  itvodes . 
sed  \ 

-e  's/\([':/]  [*:/]*\) [:/]  [*  ]*/\l/g' 

#  RTL  vectors  are  written  [...],  convert  to  #(...) 
sed  \ 

-e  's/\[/  \#\(/g'  \ 

-e  's/\]/\)/g' 

#  Squeeze  empty  lines 
cat  -s 

echo  ")" 


Figure  6.2.  From  GCC’s  output  to  SPOIL’S  input 
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This  redundant  representation  of  the  program  (three  files  coming  out  of  one 
source)  was  needed  only  to  extract  enough  information  to  map  temporary  register 
names  into  user  variables,  which  is  a  convenience  when  debugging  this  code,  but  not 
mandatory  for  this  work. 

The  activity  of  SPOIL  proper  begins  with  the  .srtl  file.  As  described  in 
Chapter  4,  the  list  of  instructions  is  successively  converted  into  a  list  of  granules,  a 
list  of  fragments,  and  a  fragment  hierarchy. 

An  initial  execution  set  hierarchy  is  also  constructed.  For  every  fragment  /  in 
the  program,  that  initial  execution  set  hierarchy  contains  either  the  execution  set 
denoted  by  (execset /)  or  the  one  denoted  by  (sloop  (execset /))  depending 
on  whether/is  a  trivial  loop. 

Some  comment  is  needed  about  these  initial  contents.  Given  that  the  execution 
set  notation  allows  one  to  express  directly  the  looping  structure  of  a  sequential  pro¬ 
gram,  one  could  expect  to  find  all  the  loops  designated  in  the  initial  hierarchy  with  a 
sloop  construct.  After  all,  determining  the  loop  structure  from  the  control  flow 
graph  is  a  well-known  problem  fASU86,  Chapter  9]. 

On  the  other  hand,  that  determination  is  nothing  more  than  the  test  for  a  very 
conspicuous  fiow  pattern,  namely  the  presence  of  a  backwards  arc.  Finding  such  pat¬ 
terns  is  the  domain  of  the  SSFG  graph  parser. 

This  means  that  SPOIL  could  have  chosen  to  identify  the  (serial)  looping  struc¬ 
ture  either  in  the  front  end  (during  standard  control  flow  analysis)  or  with  the 
transformer.  I  decided  to  leave  the  loop  detection  rules  in  the  catalog,  but  out  of  con¬ 
venience  identified  the  trivial  self  loops  (i.e.,  those  granules  that  branch  to  them¬ 
selves)  during  control  flow  graph  construction.  This  is  an  example  of  a  possible 
tradeoff  between  doing  something  in  the  front  end  and  doing  it  in  the  parallelizer. 
The  existence  of  such  tradeoffs  illustrates  the  flexibility  of  this  model  to  accommo¬ 
date  efficiency  considerations. 

All  this  preparation  leads  to  the  application  of  the  transformation  algorithm 
described  in  Chapter  5.  SPOIL’S  work  finishes  with  the  production  of  a  definitive  exe¬ 
cution  set  hierarchy  for  the  program  (including,  perhaps,  new  code  that  was  added  in 
the  course  of  exposing  parallelism)  and  the  output  of  .some  run-time  stati.stics. 

There  are  actually  two  versions  of  SPOIL.  In  one  of  them  (which  I  will  refer  to 
as  the  bblock  version),  the  granules  are  basic  blocks  of  the  RTl,  code.  In  the  other 
one  (the  cstat  version),  the  granules  correspond  roughly  to  statement  boundaries  in 
the  original  C  source.  Therefore,  the  cstat  version  operates  at  a  finer  granularity. 
The  bblock  version  was  u.sed  originally  to  test  the  general  soundness  of  the  approach 
and  to  experiment  with  mid-  to  coarse-level  parallelizations.  The  cstat  version  has 
been  used  to  reproduce  vectorization  results. 

6.3.  Costs,  Tradeoffs 

The  conceptual  clarity  obtained  by  separating  the  detection  of  parallelism  from  its 
exploitation,  and  by  encoding  parallelization  knowledge  outside  the  parallelizer  itself, 
is  gained  at  a  price.  The  enumerator  must  generate  a  number  of  subsets  of  the 
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program,  and  the  transformer  must  test  all  (and  transform  some)  of  them,  ostensibly 
without  benefit  of  architectural  constraints. 

Naturally,  this  opens  up  opportunities  for  tradeoff.  A  fragment  hierarchy  with 
small  fragments  is  going  to  explore  many  more  possibilities  than  would  one  with 
relatively  large  grain  (like  the  one  used  by  the  bblock  version  of  SPOIL),  but  can  find 
more  parallelism.  On  the  other  hand,  the  more  sub-fragments  a  fragment  has,  the 
more  aspects  will  have  to  be  inspected,  which  increases  the  total  running  time. 

First,  let’s  consider  the  question  of  size  of  the  fragment  hierarchy  used  by  SPOIL. 
As  already  explained  in  Chapter  4,  the  fragments  used  by  SPOIL  are  those  single- 
entry/single-exit  regions  of  the  control  flow  graph  that  also  happen  to  be  contiguous 
in  a  depth-first  numbering  of  the  basic  blocks,  which  1  called  dfi-fragments  in  Chapter 
4  (see  Section  4.1). 

For  general  graphs,  there  could  be  as  many  as  0(5  )  such  dfi-fragments,  where 
B  is  the  number  of  nodes  in  the  graph  (i.e.,  basic  blocks  or  C  statements  in  the  pro¬ 
gram).  On  the  other  hand,  control  flow  graphs  are  observed  to  be  a  very  restricted 
kind  of  directed  graph.  None  of  their  nodes  has  out-degree  greater  than  two.  This 
constant  bound  makes  it  likely  that  the  number  of  fragments  in  actual  programs  will 
be  more  linear  than  quadratic. 

To  test  this  hypothesis,  a  suite  of  programs  was  gathered  in  order  to  measure  the 
ratios  of  fragments  and  aspects  to  granules,  and  to  test  the  overall  coverage  of  known 


Name 

B 

F 

A 

wolfe-4.2 

6 

4 

7 

sl31 

7 

5 

9 

sl23 

7 

4 

7 

sl24 

8 

4 

7 

minmax 

9 

6 

10 

mi  nm  ax  sum 

11 

9 

16 

si  15 

11 

8 

15 

s2710 

12 

5 

9 

sploops 

14 

9 

16 

solve 

16 

14 

29 

frac 

21 

11 

20 

Steele 

37 

47 

100 

decomp 

78 

90 

201 

Table  6.1.  Medium  Grained  Test  Programs 
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parallelization  techniques.  The  programs  are  included  in  the  Appendix  B. 

Table  6.1  and  Figure  6.3  show  the  results  for  the  bblock  version.  A  total  of  13 
programs  were  considered,  ranging  in  size  from  6  to  78  basic  blocks.  In  that  table,  B 
stands  for  the  number  of  basic  blocks,  F  for  the  number  of  fragments  corresponding 
to  those  blocks,  and  A  for  the  number  of  aspects  considered  for  transformation. 

A  linear  least  squares  fit  to  the  data  in  Table  6.1  shows  the  relation 

F  =-6.01  +  1.24xB  (p=0.98)  (6.1) 

This  result  supports  the  assumption  that  fragment  count  doesn’t  grow  faster  than 
linearly  with  the  size  of  the  program,  as  the  measures  are  consistent  with  an  0[B) 
growth  rate. 

Even  given  a  linear  number  of  fragments,  we  can  still  observe  large  fragment 
hierarchies  if  there  are  too  many  aspects.  This  is  possible  due  to  the  potential 
existence  of  many  maximal  components  for  the  larger  fragments  (for  instance,  for  the 
entire  program).  Choosing  a  strategy  for  aspect  selection  is  thus  critical  to  the 
efficiency  of  this  class  of  parallelizer. 

The  test  suite  behaves  modestly  with  respect  to  aspect  formation  too.  The  linear 
least  squares  fit  (also  from  Table  6.1)  shows  the  relation 
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Fragments  (□) 
Aspects  (•) 


40  60 

Basic  Blocks 


Figure  6.3.  Medium-Grain  Experiments:  Sizes 
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A  =-16.6  +  2.79x5  (p=0.98)  (6.2) 

where  again  5  is  the  number  of  blocks  and  A  is  the  number  of  aspects. 

These  correlations  are  strong  enough  to  support  experimentally  the  claim  that 
both  the  fragment  and  the  aspect  counts  grow  only  linearly  with  the  size  of  the  pro¬ 
gram  (measured  in  basic  blocks). 

Similar  results  hold  for  the  cstat  version  of  the  experiments.  The  sizes  are  given 
in  Table  6.2.  The  relations  (represented  also  in  Figure  6.4)  are  now: 


F  =-12.82 +  3.44xG 

(p=0.99) 

(6.3) 

A  = -58.74 +  8.26XG 

(p=0.99) 

(6.4) 

where  F  and  A  designate  fragment  and  aspect  counts,  respectively,  and  G  represents 
granules  (C  statements). 

6.3.1.  Discussion 

Having  linear  amounts  (on  granule  count)  of  fragments  and  aspects  is  necessary  for 
efficiency  in  this  approach.  As  aspect  fomiation  could  potentially  construct  exponen¬ 
tial  numbers  of  aspects,  it  is  reassuring  that  real  programs  don’t  exhibit  an  explosive 
behavior. 

Indeed,  although  the  linearity  of  fragment  count  on  granule  count  was  more  or 
less  expected  from  the  constant  out-degree  of  control  flow  graphs,  there  was  no  simi- 
hu-ly  strong  hint  that  aspect  count  would  remain  linear  too.  An  imponant  open  ques¬ 
tion  is  then  the  characterization  of  fragment  and  aspect  definitions  that  ensure  modest 
growth  rates  with  respect  to  granule  count. 


Name 

G 

F 

A 

loopl 

9 

13 

23 

Ioop2 

9 

12 

20 

loop  3 

9 

12 

20 

doacr 

10 

16 

29 

pat 

14 

33 

70 

frac 

V 

130 

139 

solve 

39 

139 

329 

decomp 

134 

441 

1056 

Table  6.2.  Fine  Grained  Test  Programs 
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Figure  6.4.  Fine-Grain  Experiments;  Sizes 


Still,  the  relations  desciibed  above  don’t  imply  that  the  total  cost  of  fragment 
hierarchy  formation  will  be  linear  in  the  size  of  the  program.  SPOIL  currently  gen¬ 
erates  fragments  by  enumerating  O(G^)  pairs  of  integers,  back-mapping  from  the 
depth-first  numbering  of  the  granules,  and  then  testing  for  dfi-fragmenthood.  This 
generate-and-test  approach  Wt.s  acceptable  for  the  proof  of  concept  implementation; 
but  knowing  now  that  there  is  only  a  linear  number  of  fragments  to  be  found  suggests 
exploring  a  more  discriminating  enumeration  of  depth-first  inter\'als,  perhaps  by 
extending  known  ones  incrementally,  instead  of  enumerating  all  the  possibilities  and 
discarding  the  failed  ones.  The  question  of  whether  the  total  cost  can  be  reduced 
from  quadratic  is  still  open. 

Also,  the  programs  used  in  these  experiments  have  been  quite  small  (although 
some  of  them,  like  solve  and  decomp  are  fair  representatives  of  typical  numerical 
kernels).  It  remains  to  test  this  approach  against  much  larger  programs,  to  rule  out 
the  possibility  of  higher-order  effects  expressed  at  those  larger  sizes. 

Although  actual  running  times  don’t  provide  a  sufficiently  firm  basis  for  com¬ 
parison,  it  may  be  useful  to  mention  that  ninning  SPOIL  doesn’t  cost  excessively  in 
real  time.  The  prototype  was  instrumented  to  report  running  times  at  two  opportuni¬ 
ties:  when  the  main  data  structures  had  been  prepared  (first  fragment  and  execution 
set  hierarchies,  original  visit  order),  and  after  the  transformer  had  finished  ninning. 
The  first  time  included  time  spent  outside  the  Lisp  system,  running  GCC  twice,  and 
running  the  scripts  that  massaged  GCC’s  output.  The  trial  runs  took  (on  a  Sun 
SPARCstation  1,  the  two  temis  being  the  two  times  mentioned  above)  between 
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3. 8+7. 5  seconds  for  minmax  u.id  222.3+241.0  seconds  for  decomp.  Although  not 
industrial-quality  yet,  these  times  are  still  tolerable  in  a  prototype,  and  show  that 
there  is  reason  to  attempt  a  new  implementation  focused  on  performance. 

6.4.  Coverage  of  Transformations 

The  previous  sections  have  shown  that  the  approach  taken  by  SPOIL  is  tolerably 
efficient.  It  remains  to  be  seen  how  generally  applicable  it  is.  To  this  end,  I  will 
compare  here  SPOIL’S  catalog  to  already  known  parallelization  results,  in  order  to 
support  the  claim  that  separating  detection  from  exploitation  of  parallelism  does  not 
hinder  the  power  of  the  parallelizer. 

The  original  version  of  SPOIL  (the  bblock  version)  operates  at  a  rather  coarse 
level  of  granularity.  The  literature  has  generally  considered  fine-grain  paralleliza¬ 
tions  (vectorization,  in  particular),  so  to  compare  SPOIL  to  classical  results  one  needs 
to  refine  its  grain.  In  Section  5.4  I  have  shown  an  example  of  the  operation  of  SPOIL 
on  medium-grain  parallelizations. 

The  plan  for  the  rest  of  this  section  is  as  follow's:  first  1  compare  SPOIL'S  catalog 
with  classical  transformations  (vectorizations).  The  comparison  is  done  by  reproduc¬ 
ing  the  previous  results  in  the  languagv’  of  my  catalog.  Then  1  consider  an  especially 
hard  case,  where  the  comparison  is  against  a  prior  result  that  depended  strongly  on 
not  separating  detection  and  exploitation.  Then  I  consider  the  possible  encoding  of 
the  fairly  general  class  of  transtc.rmations  that  introduce  synchronization  to  turn  a 
serial  loop  into  a  parallel  loop.  Finally  I  consider  another  special  case,  where  both 
coarse  and  fine  grain  parallelizations  are  possible,  and  where  SPOIL  can  find  both  uni¬ 
formly. 

The  transformations  used  in  this  comparison  essentially  exhaust  the  ones  found 
in  a  careful  analysis  of  trie  literature.  Their  different  origins  and  independent 
development  made  it  hard  to  org  nize  these  comparisons  more  systematically,  so  the 
following  sections  car  give  the  "eader  the  impression  that  the.se  techniques  are  only 
haphazardly  related.  This  feeling  of  unsystematic  enumeration  of  techniques  was 
pan  of  my  motiva.ion  f  'r  'iiidertaking  this  study.  It  w'as  with  great  satisfaction  that  I 
sav,  franiework  developed  in  this  work,  thus  putting  them  in  a  context 

wheic  their  common  traits  are  clear. 

6.5.  The  Classical  Transformations 

To  test  spoil’s  competence  against  standard  transformations,  I  used  the  programs 
Icxjpl,  lcx)p2,  loop3,  Steele,  and  splcx^ps,  which  can  be  found  in  the  Appendix 
B.  The  rules  provided  in  the  catalog  (appendix  A)  were  sufficient  to  produce  the 
transfomiations  I  describe  below. 

In  Icxjpl,  the  two  statements  S  I  and  S2  happen  to  be  independent.  The  rule 
for  Loop  Distribution  converts  the  serial  loop  into  a  series  of  two  plexjps,  thereby 
catching  the  vectorization  with  a  grain-independent  rule.  This  is  one  of  the  advan¬ 
tages  of  the  approach  advocated  here:  the  detector  finds  the  opportunity  for  paralleli¬ 
zation,  and  makes  it  available  to  the  back  end  without  either  (1)  prejudicing  whether 
the  series  of  plexjps  is  the  desirable  thing  to  do,  or  (2)  (assuming  it  is)  whether  it  will 
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be  implemented  as  two  tasks,  or  as  multiple  passes  through  a  vector  engine. 

loop2  and  lopp3  show  the  use  for  the  rules  of  Partial  Distribution.  Here  we 
ask  the  data  flow  analyzer  to  tell  us  if  the  chosen  decomposition  of  the  current  frag¬ 
ment  has  a  part  that  is  loop-independent  (that  is,  its  dependences  form  a  strongly  con¬ 
nected  region  and  don’t  go  across  iterations).  In  such  case,  the  strongly  connected 
region  can  be  extracted  [P0I88,  Section  2.3.1]  and  parallelized  apart  from  the  rest. 

Notice  that  the  proposed  rules  don’t  work  if  the  separable  part  happens  to  be 
inside  the  loop,  surrounded  by  non-separable  parts.  To  have  that  work  we  need  to  ask 
of  the  front  end  that  loop  bodies  be  presented  in  topological  sorted  order  by  their 
dependences,  or  to  improve  the  matcher  (by  matching  on  the  dependence  graph,  not 
on  the  control  flow  graph).  This  limitation  has  not  affected  the  results  on  my  test 
suite,  but  it  should  be  considered  for  future  work. 

6.6.  Forcing  The  Separation:  Ignorable  Self  Anti-Dependences 

Wolfe  [Wol89,  Section  3.4]  presents  an  interesting  case.  He  considers  the  possibility 
of  just  ignoring  self  anti-dependence  cycles  by  using  knowledge  about  sequencing 
guaranteed  by  the  architecture.  For  instance,  in: 

/*  adapted  frcm  [Wol89,  Section  3.4]  */ 
for  (i  =  0;  i  <  N;  i-H-) 
a[i]  =  a[i+l]  -  1; 

one  finds  the  following  behavior:  in  the  first  iteration,  a[0]  gets  assigned  a[l], 
which  itself  gets  modified  in  the  next  iteration  by  a  [2] ,  etc. 

So,  in  appearance,  we  cannot  reorganize  the  loop  for  parallelization,  due  to  the 
“clocked”  behavior  of  the  moves  to  a. 

However,  if  we  knew  that  N  is  less  than  the  number  of  vector  registers  in  a 
machine  that  guarantees  to  perfomt  all  the  vector  loads  before  any  of  the  stores,  then 
it  would  seem  we  could  go  ahead  and  parallelize  the  loop  in  spite  of  the  obvious 
dependences.  Of  course,  if  we  don’t  know  the  value  of  N  or  its  relation  to  the  vector 
width,  we  can  always  do  loop  blocking. 

The  challenge  to  the  method  advocated  here  is  that  the  success  of  the  vectoriza- 
tion  depends  on  (implicit)  guarantees  on  the  order  of  loads  and  stores. 

Can  we  do  the  same  without  breaking  the  separation  between  detection  and 
exploitation  of  piu'allelism? 

It  turns  out  that  it  is  indeed  possible.  Consider  the  rule  for  ignorable  self¬ 
antidependence  cycles  from  appendix  A,  repeated  here  for  convenient  reference: 

(catalog. add-rule  ;wolfe  p.64 

cat 

"Ignorable  self-antidependence  cycles" 

(guard. new  ' (:and 

(: exist s-execset  ?f  (:sloop  (:execset  ?fb) ) ) 

(: flow- relation  ?f  ?fb  ?fb  : flow-dependent) 

(:not  (:  flow-relation  ?f  ?fb  ?fb  :  anti-dependent) ) 


68 


(:not  (:  flow-relation  ?f  ?fb  ?fb  :  output -dependent ) ) 

(:  flow-attribute  ?fb  ()  ?def-locs  ?d) 

( :match  ?d  (?target) ) 

(:code  ?fb  ?oldcode))) 

(transform. new  '  (: apply 

(rnew-name  ?new-array) 

(:new-code  ( (call-insn  'copy  Ttarget  ?new-array) ) 
nil  ?copy-in) 

( :with-new-name  Ttarget  Tnew-array  Toldcode  Tnewcode) 
( :new-code  Tnewcode  Tnewbody) 

(:new-code  ((call-insn  'copy  Tnew-array  Ttarget)) 
nil  Tcopy-out) 

( : new-execset  ( : series 
Tccpy-in 

( rplocp  Tnevtoody) 

Tcopy-o\it) 

Tnew-eset) 

( :add-to-execset  Tf  Tnew-eset)))) 

What  this  nile  does  is  to  convert  the  implicit  guarantee  of  serialization  into  an 
explicit  one,  by  introducing  a  temporary  array  to  hold  the  new  left-hand  sides  while 
the  old  ones  are  still  being  used.  The  back  end  can  now  be  informed  (perhaps  by 
introducing  a  storage  class  to  this  effect)  of  the  caching  nature  of  the  temporary  that 
the  front  end  has  introduced.  If  the  back  end  maps  the  temporary  into  the  vector 
legisters.  it  would  have  eliminated  the  actual  copies  and  allocation,  and  it  can  then 
exploit  knowledge  about  the  machine.  On  the  other  hand,  if  no  such  guarantees  are 
given  by  the  hardware,  but  the  copies  happen  to  be  cheap  due  to  any  other  reason,  the 
back  end  has  available  a  locus  of  parallelism  that  could  be  useful,  but  that  in  no 
opportunity  is  incorrect  (because  its  conditions  for  correctness  are  enforced  expli¬ 
citly). 

6.7.  DOACROSS  -  A  General  Class  of  Loop  Parallelization 

Here  I  consider  a  very  general  class  of  transformations  that  add  synchronization 
operations  to  serial  loops  in  order  to  run  them  in  parallel,  yet  honor  the  dependences. 

The  reference  here  is  program  doacr,  from  [Wol89,  p.  76].  The  critical  points  are 
these: 

(1)  We  can  (at  least  in  the  cstat  version)  query  for  the  iteration  distances  between 
statements  in  the  loop.  Conceptually,  we  can  ask  the  data  flow  analyzer  to 
provide  us  with  the  intermediate  ccxle  list  needed  to  replace  the  original  one 
to  secure  synchronization.  Producing  and  altering  intermediate  code  lists  is 
in  the  domain  of  the  front  end  so  this  requires  no  special  purpose  mechanisms 
other  than  an  interface  to  request  these  lists  when  needed.  This  view  is 
expressed  in  the  following  rule: 

(catalog . add- rule 
cat 


;wolfe/doacr 


69 


"DQACROSS" 

(guard. new  '  (rand 

(: exist s-execset  ?f 

(: sloop  .  ?loqpbody)) 

(:  match  ?c  (?body)) 

(: exist s-execset  ?body 

( : series  .  ?loqpbody) ) 
(.•flow-attribute  ?f  (?body) 

:  synchronized-code  ?synch-code) ) ) 

(transform. new  '  (: apply 

(rmap  (:  apply 

( : match  ?synch  (?fid  ?newcode) ) 
(:new-code  ?newcode  ?fid  ?nfid) 

( :new-execset  (:execset  ?nfid)  ?enfid)) 
?synch  ?synch-code  ?enfid  ?enfids) 

( rcorbine-execsets  rploqp  ?enfids  ?new-eset) 

( :add-to-e3:ecset  ?f  ?new-eset) ) ) ) 

This  rule  essentially  depends  on  there  being  a  data  flow  analyzer  for  the  attribute 
:  synchronized-code.  This  is  consistent  with  our  previous  observation  that  some 
transformadons,  even  when  doable  in  the  parallelizer  (we  could  have  asked  for  the 
iteration  distances  ourselves,  and  depended  on  the  back  end  to  eliminate  redundant 
waits  and  posts),  should  be  actually  moved  to  the  front  end  (because  they  are  easier 
there)  or  to  the  back  end  (because  they  are  architecture-dependent) 

6.8.  Operating  Uniformly  at  More  Than  One  Grain 

I  observed  above  (while  discussing  the  loopl  example),  that  some  rules  can  capture 
the  available  parallelism  of  a  fragment  very  cleanly,  independently  of  the  granularit'/ 
of  the  tasks  in  the  target  machine.  In  loopl  it  turned  out  that  the  same  rule 
discovered  what  can  be  called  either  a  concurrentization  or  a  vectorization  (depend¬ 
ing  on  whether  one  can  add  two  vectors  into  a  third,  a  commonly  available  operation) 
with  just  one  test.  A  similar  thing  can  be  observed  in  the  pat  example  (reproduced 
below): 

/*  Adapted  from  Smith  and  Appelbe,  ICCP  1988, 

*  '  'PAT  —  An  Interactive  Fortran  Parallelizing  Assistant  Tool' ' 

*/ 

int 

main  0 

{ 

float 
int 

extern  void 


init  0  ; 


a  [20],  b[20],  sum,  max; 

j; 

initO  ; 
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stm  =  0; 

j  =  20; 

for  (i  =  0;  i  <  20;  i++)  { 
sum  ■=  sum  +  a[i]; 
if  (i  >  0) 

a[i]  =  b[j]  +  a[i  -  1]; 
if  (b[i]  >  max) 
max  =•  bi.i]  ; 
j  =  j  -  1; 


printf("sum  :  %f9.5  max  :  %f8.50,  sum,  max); 


This  program  is  a  C  version  of  the  pat.f  program  (both  in  appendix  B).  The 
original  appeared  in  [SmA88,  Diagram  1].  The  parallelization  of  interest  here  intro¬ 
duces  explicit  synchronization  in  the  loop  body.  Notice  that  this  program  shares  with 
the  minmax  example  the  possibility  of  distributing  the  sum  and  the  max  computation, 
and  to  partially  distribute  the  vector  shifted  addition  in  the  middle.  SPOIL’S  catalog 
would  apply  all  of  these  transfomiations,  and  present  them  as  options  to  the  back  end, 
while  PAT  apparently  concentrates  on  the  coarser-level  transformations.  It  is 
interesting  to  notice  that,  while  SPOIL  provides  choices  to  the  back  end,  PAT  is 
designed  to  provide  interactiv<’  choices  to  the  user.  One  could  imagine  a  merge  of  the 
two  approaches. 

6.9.  Discussion 

My  experience  with  SPOIL  has  shown  that  suitable  definitions  of  granule,  fragment, 
and  aspect  permit  a  successful  implementation  of  the  already  known  parallelizations. 

With  respect  to  practical  applicability,  the  most  important  feature  was  the  selec¬ 
tion  of  aspects.  One  of  my  early  tries  involved  a  medium  grain  analysis  (granules 
were  basic  blocks)  and  a  rich  fragment  hierarchy.  Aspects  were  fomred  by  fitting  any 
SLibfragnients  to  the  residual  masses  after  one  collapse,  thereby  matching  ever}'  maxi¬ 
mal  component  with  all  the  components  (no  matter  how'  small)  disjoint  to  it.  The 
hope  was  that  there  would  be  net  too  many  aspects  after  all. 

Soon  it  was  clear  that  that  wasn’t  the  case.  Although  very  small  programs 
seemed  to  show  a  linear  cost,  there  were  a  few  examples  (like  the  decotp  program) 
that  required  seemingly  exponential  numbers  of  aspects.  This  happens  whenever  the 
entire  program  contains  many  “small”  fragments. 

Two  possible  cures  were  tried.  The  first  was  to  avoid  all  the  single-granule  frag¬ 
ments,  except  for  the  self-loops.  Although  this  improved  conditions  (and  can  be 
defended  in  terms  of  quality  of  abstraction:  single-granule  fragments  don’t  abstract 
any  structure),  it  wasn’t  sufficient  to  yield  a  practical  fragment  hierarchy. 

The  second  attempted  cure  was  to  try  only  the  maximal  components  when 
decomposing  a  fragment.  The  combination  of  this  and  the  first  approach  yields  the 
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linear-size  hierarchies  discussed  above.  But  this  severe  pruning  of  the  possible 
aspects  could  hide  useful  parallelizations.  What  the  examples  above  show  is  that  that 
is  happily  not  the  case.  So,  it  is  possible  to  enjoy  the  intellectual  leverage  obtained 
by  performing  the  recommended  separation  of  concerns,  while  not  renouncing  any  of 
the  generally  useful  parallelizations. 

These  experiments  have  established  my  confidence  in  the  applicability  of  the 
thesis  of  this  work.  SPOIL  has  been  tested  successfully,  both  in  terms  of  coverage  of 
previous  work  and  of  indication  of  practical  efficiency,  without  so  far  showing  any 
serious  flaw  in  the  model  (as  opposed  to  shortcomings  in  the  prototype).  Within  its 
proof-of-concept  scope,  SPOIL  supports  the  claims  of  this  dissertation. 


CHAPTER  7 


Conclusions 


7.1.  Summary 

It  is  time  to  look  back  at  the  claims  put  forth  in  this  v.ork,  and  to  summarize  what  has 
been  learned  in  the  process. 

The  object  of  this  study  has  been  the  parallelization  of  ordinary  code.  By  such,  1 
mean  imperative  code  with  sequential  semantics,  further  constrained  by  the  need  to 
represent  accurately  the  control  and  data  flow'  relationships.  1  havv  claimed  that  the 
total  task  of  parallelization  is  intellectually  more  manageable  if  one  separates  the 
concerns  of  detection  and  exploitation  of  parallelism.  This  investigation  has  concen¬ 
trated  on  the  organization  of  the  detection  component,  aiming  to  support  the  larger 
claim  about  parallelization  in  general. 

It  is  easy  to  believe  that  keeping  apart  two  demanding  purposes,  as  detection 
and  exploitation  of  parallelism  are,  has  to  result  in  somewhat  cleaner  approaches  to 
each  of  them.  What  is  not  immediately  clear  is  whether  this  separation  is  possible,  or 
whether  it  will  introduce  problems  harder  than  those  solved  by  its  means. 

I  believe  that  the  preceding  pages  have  shown  how  indeed  it  is  possible  to  keep 
the  two  concerns  separate.  The  main  insight  is  to  make  sure  that  a  explicitly  parallel 
representation  (like  the  one  provided  by  the  execution  st;  expressions)  be  used  as  the 
interface  between  the  two  problems.  To  recapitulate,  the  need  for  explicit  parallelism 
in  the  interface  is  a  reflection  of  the  need  of  the  back  end  for  positive  infomiation 
about  parallelism.  The  standard  dependence  representations  are  good  at  telling  us 
what  can't  be  done  in  parallel.  It  is  not  immediately  obvious  how  one  goes  from  that 
fonn  of  negative  information  to  figuring  out  w  hat  can  be  done  in  parallel. 

So,  I  have  described  a  general  schema  of  program  transfomiation,  that  yields  an 
explicitly  parallel  representation. 

This  schema  (the  highest  level  of  abstraction  discussed  in  the  introduction.  Sec¬ 
tion  1.3)  can  be  characterized  by  two  properties. 

The  first  is  a  hierarchical  description  of  the  program,  which  provides  us  with  the 
means  of  abstraction  needed  to  eliminate  extraneous  features  from  consideration, 
while  permitting  us  to  arrive  at  the  parallelization  of  large,  possibly  complicated, 
pieces  of  code  by  composing  the  results  of  the  analysis  of  their  subparts. 

The  second  is  another  separation  of  concerns.  The  model  distinguishes  the  con¬ 
trol  structure  used  to  enumerate  pans  of  the  hierarchy  from  the  specific  transforma¬ 
tions  that  expose  larger  execution  sets.  This  separation  permits  us  to  capture  the 
observation  that  ‘perfect’  parallelization  is  generally  undecidable  (by  reduction  from 
halting  [Ber66J),  while  being  able  to  reason  that  a  given  parallelizer,  working  within 
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polynomial  resource  bounds,  won’t  miss  any  parallelization  it  ‘should’  find. 

The  experimental  evidence  accumulated  via  SPOIL  confirms  the  applicability  of 
this  approach. 

So,  when  wanting  to  write  a  specific  parallelizer,  I  would  proceed  this  way  (and 
this  dissertation  argues  that  so  should  you): 

(1)  First  of  all,  I  w'ould  figure  out  a  unit  of  resolution  for  the  analysis.  Suitable 
candidates  are  individual  register  transfers,  statements  in  the  language  (at 
least,  assignments),  or  basic  blocks.  These  are  the  granules  of  the  analysis. 
This  unit  of  resolution  imposes  a  bound  on  how  thorough  will  be  the  search 
for  parallelism,  but  simultaneously  provides  a  way  to  control  the  resources 
spent  in  that  search.  Granules  are  also  the  unit  of  parallel  execution.  The 
number  of  granules  in  a  program  will  be  considered  its  size.  A  constraint  on 
our  freedom  to  define  granules  is  that  there  should  be  a  practical  way  to 
describe  data  and  control  flow  relationships  among  granules. 

(2)  Second,  I  would  consider  a  hierarchical  structure  on  program  parts.  At  tlie 
lowest  level  of  the  hierarchy  we  have  some  or  all  of  the  granules.  Then,  at 
higher  levels,  w-e  have  control  flow  structures  composed  of  elements  of  the 
lower  levels.  Call  the  elements  of  this  hierarchy  fragments.  Two  conditions 
restrain  our  freedom  to  define  fragments:  (1)  the  cost  of  going  from  granules 
to  fragments  should  be  bound  by  a  polynomial  on  the  size  of  the  program, 
and  (2)  there  should  be  a  convenient  way  to  infer  data  and  control  flow  rela¬ 
tionships  among  fragments  from  the  relationships  know-n  to  hold  among 
granules.  (Most  of  the  elaboration  so  far  has  counterparts  in  previous  paral¬ 
lelization  work,  the  contribution  here  being  having  put  them  in  a  common 
framew'ork  sufficiently  generic  to  encompass  the  previous  approaches.) 

(3)  Third,  1  would  consider  “quotients”  of  fragments  w'ith  respect  to  sets  of  their 
subsets.  By  a  “quotient”  here  1  mean  the  collapse  of  the  internal  structure  of 
the  named  subsets,  such  that  they  appear  as  single  nodes  for  the  purposes  of 
this  analysis.  A  fragment,  considered  along  with  the  collapse  of  some  of  its 
subfragments,  is  what  1  call  an  aspect.  (The  notions  of  aspect  and  of  driving 

analysis  by  aspects,  are,  as  far  as  1  know,  a  novelty  of  this  w-ork.)  Of 
coarse,  the  same  restriction  applies  here:  the  cost  of  enumerating  all  the 
aspects  should  be  bound  by  a  polynomial  on  the  number  of  fragments 
(hence,  on  the  number  of  granules). 

(4)  Validating  all  those  efficiency  constraints  may  be  difficult,  especially  if 
appealing  definitions  have  worst-case  bounds  that  are  not  polynomial  (or  are 
polynomials  of  high  degree,  where  for  practical  purposes  anything  above 
degree  2  or  3  is  probably  too  high).  One  possibility  (shown  in  this  w'ork)  is  to 
use  standard  statistical  techniques  on  real  programs,  in  the  effort  to  figure  out 
if  the  worst-case  presents  itself  in  practice. 

(5)  Now,  I  would  organize  the  enumeration  of  aspects  of  fragments  in  such  a  way 
that  one  can  eventually  use  in  the  parallelization  of  a  fragment  the  results  of 
having  parallelized  its  subparts.  This  is  the  eventual  enumeration  ordering 
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property  of  the  analysis.  This  property  guarantees  that,  after  inspecting  a 
fragment  for  the  last  time,  all  of  its  super-fragments  will  still  have  at  least  one 
chance  to  be  inspected  (and,  therefore,  to  exploit  the  previous  results). 

(6)  Having  organized  this  fixed  search  strategy  (i.e.,  defining  what  granules,  frag¬ 
ments  and  aspects  will  be,  and  providing  iterators  that  satisfy  the  eventual 
enumeration  ordering  property),  I  would  then  design,  separately,  the  rules  of 
transformation  to  apply.  To  this  end,  I  would  consider  the  characterization  of 
the  fragments  in  terms  of  the  data  and  control  flow  relationships,  and  express 
conditions  to  convert  a  piece  of  code  into  another  with  a  larger  execution  set. 
The  conditions  are  to  be  explicit,  and  assume  nothing  about  the  target 
machine.  This  is  generally  possible,  as  their  correctness  depends  only  on  the 
semantics  of  the  underlying  language.  The  collection  of  these  conditions  and 
transformations  is  the  catalog. 

Experience  with  SPOIL  has  shown  that  one  can  conveniently  put  in  the  catalog 
transformations  that  either  (1)  don’t  increase  any  execution  sets,  or  (2)  exploit 
knowledge  of  target  architecture.  My  first  reaction  would  be  to  banish  the  first  to  the 
front  end  (and  then  define  a  condition-and-transformation  language  that  let  me  query 
the  front  end  when  needed),  and  the  second  to  the  back  end,  where  they  properly 
belong.  However,  although  my  model  sheds  light  on  w'hich  transfomiations  are  prop¬ 
erly  called  parallelizations,  it  is  to  be  expected  that  practical  implementations  will 
compromise  here  and  tolerate  some  blurring  of  this  distinction,  on  account  of 
efficiency. 

The  same  is  true  of  the  standard  compiler  model,  and  those  compromises  (for 
instance,  the  merging  of  lexical  analysis  and  parsing  in  one  phase)  are  known  not  to 
hinder  the  use  of  the  standard  model  to  understand  or  design  such  compilers.  It 
seems  to  me  that  the  model  presented  in  this  dissertation,  because  of  the  flexibility 
involved  in  the  various  entities  that  need  to  be  defined  in  the  course  of  its  use,  will 
show  the  same  robustness  in  the  presence  of  compromises  dictated  by  a  pragmatic 
sense. 

On  the  other  hand,  that  very  same  flexibility  makes  it  difficult  to  present  testable 
claims  about  the  adequacy  of  my  model  for  actual  parallelization  work.  After  all, 
most  program  transformation  schemes  (even  sequential  compilation!)  could  be  cast  in 
terms  close  to  the  language  of  granules,  fragments,  aspects,  etc.  For  this  reason  I 
presented  specific  instantiations  of  these  concepts.  SPOIL  represents  programs  as 
graphs  of  either  C  statements,  or  of  basic  blocks.  In  addition,  I  provided  a  specific 
definition  of  fragment  (in  temis  of  depth-first-numbering  intervals)  and  a  specific 
definition  of  aspect  (in  tenns  of  quotients  of  fragments  with  respect  to  their  maximal 
subfragments).  TTien  I  provided  an  instantiation  of  the  catalog  notion,  that  uses  only 
specific  primitives  of  limited  power  (thereby  not  equivalent  to  arbitrary  code).  The 
combination  of  these  instantiations  is  seen  to  be  efficiently  practical  (at  least  in  the 
statistical  average  case),  and  also  capable  to  express  the  parallelizations  in  the  litera¬ 
ture.  Indeed,  the  version  of  SPOIL  that  uses  C-level  statements  is  seen  to  be  able  to 
discover  parallelism  at  both  fine  and  coarse  grains  simultaneously,  precisely  because 
it  is  not  unduly  concerned  with  fitting  a  given  architecture. 
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This  evidence  reinforces  my  claim  that  the  general  model  exhibits,  in  its  domain 
of  competence,  similar  descriptive  and  prescriptive  powers  as  the  standard  model 
does  in  its  own. 

7.2.  Future  Work 

Many  questions  have  remained  open,  or  have  been  newly  opened,  in  the  course  of 
this  investigation.  This  section  collects  some  of  them. 

First  of  all,  a  definitively  persuasive  demonstration  of  the  usability  of  my  model 
has  to  go  beyond  detection  of  parallelism,  and  enter  into  the  exploitation  domain.  As 
in  the  sequential  case,  I  would  expect  that  the  synthesis  of  runnable  code  be  less 
amenable  to  a  direct,  theoretically  principled  attack,  than  the  analysis  has  shown 
itself  to  be.  Still,  I  would  like  to  investigate  ways  to  map  information  in  the  execu¬ 
tion  set  hierarchy  (plus  the  rest  of  the  program  representation)  to  runnable  code. 

There  is  also  work  to  do  in  the  front  end.  SPOIL  does  a  very  light  job  of  data 
flow  analysis,  and  the  current  implementation  still  uses  queries  to  the  user  to  answer 
some  of  the  traditional  data  flow  questions.  Replacing  the  user  with  a  real  data  flow 
analyzer  would  permit  a  better  assessment  of  the  eftectiveness  with  which  SPOIL 
exploits  the  available  information.  Also,  I  would  like  to  determine  how  far  can  a  sim¬ 
ple  catalog  go  when  the  data  flow  analyzer  cannot  obtain  precise  estimations  because 
of  aliasing. 

Another  point  of  interest  is  the  possibility  of  using  one  of  the  contemporary 
combined  representations  of  control  and  data  flow  relationships  [CFR88,FOW87] 
instead  of  traditional  control  flow  graphs  augmented  with  data  flow  information, 
thereby  further  unifying  the  analysis  and  reducing  the  number  of  primitives  needed  in 
the  catalog. 

Another  area  of  investigation  is  the  exploration  of  other  definitions  of  granule, 
fragment  and  aspect,  with  views  to  characterize  the  instantiations  of  these  concepts 
that  are  effective  in  practice. 

There  is  also  the  issue  that  SPOIL’S  current  catalog  is  flat:  each  rule  is  tried  in 
order  for  every  aspect.  Many  subtests  are  repeated  for  the  same  aspect.  A  structured 
catalog  would  cut  down  substantial!)'  on  the  cost  of  the  analysis.  It  also  opens  up  the 
question  (well  known  in  program  optimization)  of  detemfining  optimal  rule  order¬ 
ings.  Perhaps  .some  non-detemiinistic  presentation  of  the  guards  w'ill  be  needed. 

Finally,  the  work  discussed  here  belongs  in  the  long  tradition  of  program 
representation  and  analysis.  I  am  interested  in  knowing  whether  it  is  possible  to  go 
from  a  formal  representation  of  the  semantics  of  a  language  to  an  automatic  choice  of 
definitions  for  granule,  fragment,  etc... 


Bibliography 


[ASU861 

fAik88] 

[A1C72] 

[A1C761 

[AlK82i 

[AKP831 

IABC871 

[ACK871 

[A1J88] 

[AmS861 


A.  V.  AHO,  R.  Sethi  and  J.  D.  Ullman,  Compilers:  Principles, 
Techniques  and  Tools,  Addison-Wesley  Publishing  Company,  1986. 

A.  AIKEN,  “Compaction-Based  Parallelization”,  Ph.D.  Thesis,  Tech. 
Rep.  88-922,  Department  of  Computer  Science,  Cornell  University, 
Ithaca,  NY  14853-7501,  June  1988. 

F.  E.  Allen  and  J.  Cocke,  “A  Catalogue  of  Optimizing 
Transformations”,  in  Design  and  Optimization  of  Compilers,  R.  RUSTIN’ 
(editor),  Prentice  Hall,  Englewood  Cliffs,  NJ,  1972,  1-30.  Courant 
Computer  Science  Symposium  5,  May  1971. 

F.  Allen  and  J.  Cocke,  “A  Program  Data  Flow  Analysis  Procedure”, 
Comm,  of  the  ACM  19,  3  (March  1976),  137-147,  ACM. 

J.  R.  Allen  and  K.  Kennedy,  “PFC:  a  Program  to  Convert  Fortran  to 
Parallel  Form”,  in  Proceedings  IBM  Conf  Parallel  Computers  in 
Scientific  Computations,  Flome,  1982. 

J.  R.  ALLEN,  K.  KENTJEDY,  C.  PORTERFIELD  and  J.  WARREN, 
“Conversion  of  Control  Dependence  to  Data  Dependence”,  Conference 
Record  of  the  Tenth  ACM  Symp.  on  Prin.  of  Prog.  Lang.,  Austin,  Texas, 
Januar>'  1983,  177-189. 

F.  ALLEN,  M.  BURKE,  P.  CHARLES,  R.  CYTRON  and  J.  FERRANTE,  “An 
Overview  of  the  PTRAN  Analysis  System  for  Multiprocessing”,  13115 
(#56866),  Computer  Science  Department,  IBM  T.  J.  Watson  Research 
Center,  Yorktown  Heights,  NY  10598,  29  September  1987. 

R.  ALLEN,  D.  Callahan  and  K.  Kennedy,  “Automatic 
Decomposition  of  Scientific  Programs  for  Parallel  Execution”, 
Conference  Record  of  the  Fourteenth  ACM  Symp.  on  Prin.  of  Prog. 
Lang.,  Munich,  West  Germany,  January’  1987,  63-16. 

R.  Allen  and  S.  Johnson,  “Compiling  C  for  Vectorization, 
Parallelization,  and  Inline  Expansion”,  SIGPLAN  Notices: 
Proceedings  of  the  SIGPLAN'88  Conference  on  Programming 
Language  Design  and  Implementation  23,  1  (July  1988),  241-249, 
ACM. 

A.  Amir  and  C.  H.  Smith,  “The  Syntax  of  Parallelism”,  UMIACS- 
Tech.  Rep. -86-24,  Institute  for  Advanced  Computer  Studies,  University 
of  Mary'land,  November  1986.  AlsoCS-Tech.  Rep. -1744,  Department  of 
Computer  Science. 


76 


77 


[AAG87] 

|Ban79] 

[Ber66] 

[Bur87] 

[Cal88] 

[CaH74] 

lCam76] 

IC0K88I 

(CoT90| 


|Cro91| 


|CyF881 


ICFR88I 


M.  annaratone,  e.  arnould,  t.  Gross,  h.  t.  kung,  m.  Lam,  o. 
MenzilcIOGLU  and  J.  A.  WEBB,  “The  Warp  Computer:  Architecture, 
Implementation  and  Performance’’,  Transactions  on  Computers  C-36,  2 
(December  1987),  IEEE. 

U.  Banerjee,  “Speedup  of  Ordinary  Programs’’,  UIUCDS-R-79-989,  U 
of  Illinois  at  Urbana-Champaign,  October  1979. 

A.  J.  Bernstein,  “Analysis  of  Programs  for  Parallel  Processing’’,  /FEE 
Transactions  on  Electronic  Computers  EC-15,  5  (1966),  757-763,  IEEE. 

M.  Burke,  “An  Interval-Based  Approach  to  Exhaustive  and 
Incremental  Interprocedural  Data  Flow  Analysis’’,  12702  (#56865), 
Computer  Science  Department,  IBM  T.  J.  Watson  Research  Center, 
Yorktown  Heights,  NY  10598,  1  September  1987. 

D.  Callahan,  “The  Program  Summary  Graph  and  Flow-sensitive 
Interprocedural  Data  Flow  Analysis’’,  SIGPLAN  Notices:  Proceedings 
of  the  SIGPLAN' 88  Conference  on  Programming  Language  Design 
and  Implementation  23, 1  (July  1988),  47-56,  ACM. 

R.  H.  Campbell  and  N.  Habermann,  “The  specification  of  process 
synchronization  by  Path  Expressions’’,  in  LNCS,  vol.  16  ,  Springer- 
Verlag,  1974. 

R.  H.  Campbell,  Path  Expressions:  A  technique  for  specifying  process 
synchronization,  Ph.D.  Dissenation,  Comput.  Lab.,  University  of 
Newcastle-Upon-Tyne,  August  1976. 

K.  D.  Cooper  and  K.  Kennedy,  “Interprocedural  Side-Effect  Analysis 
in  Linear  Time’’,  SIGPLAN  Notices:  Proceedings  of  the  SIGPLAN’88 
Conference  on  Programming  Language  Design  and  Implementation 
23, 1  (July  1988),  57-66,  ACM. 

W.  R.  Cowell  and  C.  P.  Thompson,  “Tools  to  Aid  in  Discovering 
Ptu-allelism  and  Localizing  Arithmetic  in  Fortran  Programs’’, 
Software— Practice  and  Experience  20,  1  (January  1990),  25-47,  John 
Wiley  &  Sons. 

L.  A.  CrOWL,  “Architectural  Adaptability  in  Parallel  Programming’’, 
Tech.  Rep.  381,  University  of  Rochester,  Department  of  Computer 
Science,  Ph.D.  Thesis,  May  1991. 

R.  CytRON  and  J.  FERRANTE,  “An  Improved  Control-Dependence 
Algorithm’’,  RC  13291  (#60163),  IBM  Research  Division,  T.  J.  Watson 
Resetirch  Center,  Yorktown  Heights,  NY  10598,  January  1988. 

R.  CYTRON,  J.  FERRANTE,  B.  K.  ROSEN,  M.  N.  WEGMAN  and  F.  K. 
ZADECK,  “An  Efficient  Method  of  Computing  Static  Single  Assignment 
Fomi’’,  CS-88-16,  Brown  University,  Depanment  of  Computer  Science, 
Providence,  Rhode  Island  02912,  October  1988.  Also  in  16th  ACM 
Symp.  on  Prin.  of  Prog.  Lang.,  1989. 


78 


[DaF84] 

[E11861 

[FKZ761 

[Fe0831 

1FOW871 

[Fis83] 

[FoR72J 

[Gil58] 

lGol87] 

lHar86] 

lHeU72| 

[HeU75] 

fHRB88l 

[Jed87| 


J.  W.  Davidson  and  C.  Fraser,  “Register  Allocation  and  Exhaustive 
Peephole  Optimization’’,  Software — Practice  &  Experience  14,  9 
(September  1984),  857-866,  John  Wiley  &.  Sons. 

J.  R.  Ellis,  Bulldog:  A  compiler  for  VUW  Architectures,  MIT  Press, 
1986.  Also  Tech.  Rep.  YALEU/DCS-RR-364.  Originally  a  Yale 
University  Ph.D.  Thesis,  February  1985. 

R.  FARROW,  K.  KENNEDY  and  L.  ZUCCONl,  “Graph  Grammars  and 
Global  Program  Data  Flow  Analysis’’,  Proc.  17‘^  Annual  Symp.  on 
Foundations  of  Computer  Science,  Houston,  1976,  42-56. 

J.  FERRANTE  and  K.  J.  OTTENSTEIN,  “A  Program  Form  Based  on  Data 
Dependency  in  Predicate  Regions’’,  Conference  Record  of  the  Tenth 
ACM  Symp.  on  Prin.  of  Prog.  Lang.,  Austin,  Texas,  January  1983, 
217-236. 

J.  FERRANTE,  K.  J.  OTTENSTEIN  and  J.  D.  WARREN,  “The  Program 
Dependence  Graph  and  Its  Use  in  Optimization’’,  Trans.  Prog.  Lang  and 
Systems  9,  3  (July  1987),  319-349,  ACM. 

J.  A.  Fisher,  “Ver}’  Long  Instruction  Word  Architectures  and  the 
ELI-512’’,  Tenth  Annual  Symposium  on  Computer  Architecture,  June 
1983,  140-150. 

C.  C.  Foster  and  E.  M.  RISEMAN,  “The  Inhibition  of  Potential 
Piirallelism  by  Conditional  Jumps’’,  IEEE  Transactions  on  Computers 
21,  12  (1972),  1405-1411,  IEEE. 

S.  GILL,  “Parallel  Programming’’,  The  Computer  Journal  I  (April 
1958),  2-10,  The  British  Computing  Society. 

R.  GoldblATT,  Logics  of  Time  and  Computation,  Center  for  the  Study 
of  Language  and  Infomiation,  1987. 

W.  L.  Harrison,  “Compiling  Lisp  for  Evaluation  on  a  Tightly 
Coupled  Multiprocessor’’,  CSRD  Rpt.  No.  565,  Center  for 
Supercomputing  Research  and  Development,  University  of  Illinois  at 
Urbana-Champaign,  Urbana,  IL  61801-2932,  20  March  1986. 

M,  S.  IIECHT  and  J.  D.  ULLMAN,  “Flow  Graph  Reducibility’’,  SIAM  J. 
Compiit.  I,  2  (June  1972),  188-202. 

M.  S,  HECHT  and  J.  D.  ULLMAN,  “A  Simple  Algorithm  for  Global  Data 
Flow  Analysis’’,  .SIAM  Journal  on  Computing  4,  4  (December  1975), 
519-532,  SIAM, 

S.  HORWITZ,  T.  Reps  and  D.  Binkley,  “Imerprocedural  Slicing  Using 
Dependence  Graphs’’,  SIGPLAN  Notices:  Proceedings  of  the 
SI G PLAN' 88  Conference  on  Programming  Language  Design  and 
Implementation  23,  7  (July  1988),  35-46,  ACM. 

J.  JEDRZEJOWICZ,  “Nesting  of  Shuffle  Closure  Is  Important’’, 
Information  Processing  Letters  25  (26  July  1987),  363-367,  North- 


79 


IJed88] 

[Kas75] 

[Kas73] 

[KenSOj 

fKeR78] 

[KeR89| 

fKuc75] 

lKuM84] 

iLam881 

[Lam891 

(Lam8(»| 

fLar89I 

ILBR81I 

[Li88I 


Holland  Publishing  Company. 

J.  JedrzejOWICZ,  “Infinite  Hierarchy  of  Expressions  Containing  Shuffle 
Closure  Operator”,  Information  Processing  Letters  28  (30  May  1988), 
33-37,  North-Holland  Publishing  Company. 

V.  N.  Kas’JANOV,  “Distinguishing  Hammocks  in  a  Directed  Graph”, 
Soviet  Math.  Doklady  16,  5  (1975),  448-450,  American  Mathematical 
Society. 

V.  N.  Kasyanov,  “Some  Properties  of  Fully  Reducible  Graphs”, 
Information  Processing  Letters  2  (1973),  113-117,  North-Holland 
Publishing  Company. 

K.  Kennedy,  “'Automatic  translation  of  Fortran  programs  to  vector 
form”,  476  029  4,  Rice  U.,  October  1980. 

B.  W.  KERNIGHAN  and  D.  M.  RITCHIE,  The  C  Programming  Language, 
Prentice-Hall,  1978. 

B.  W.  KernighaN  and  D.  M.  RlTCHlE,  The  C  Programming  Language, 
Prentice-Hall,  1989.  Second  Edition. 

D.  J.  KUCK,  “Parallel  Processing  of  Ordinary  Programs”,  UIUCDCS- 
R-75-767,  University  of  Illinois  at  Urbana-Champaign  Dept,  of  C.  Sc., 
November  1975. 

H.  T.  KUNG  and  O.  MENZlLCIOGLU,  Design  Specifications  for  the  CMU 
Warp  Proressor,  Camegie-Mellon  University,  1984. 

M.  Lam,  “Software  Pipelining;  An  Effective  Scheduling  Technique  for 
VLIW  Machines”,  SIGPLAN  Notices:  Proceedings  of  the  SIGPLAN'88 
Conf  mcnce  on  Programming  Language  Design  and  Implementation 
23, 1  (July  1988),  318-328,  ACM. 

M.  S  Lam,  a  Systolic  Array  Optimizing  Compiler,  Kluwer  Academic 
Pubhshen;,  1989. 

L.  Lamport,  “The  Mutual  Exc  usion  Problem  Part  I  —  A  Theory  of 
Interprocess  Communication”,  ACM  33.  2  (April  1986),  331-326, 
/,CM. 

J.  K.  LARUS.  Restructuring  Symbolic  Programs  for  Concurrent 
Lxtcuhon  on  Multiprocessc  Computer  Science  Division  (EECS), 
University  oi  Berkeley,  Berkeley,  CA  94720,  May  1989.  Ph.D.  Thesis; 
also  report  UCB/Compctci  Science  Dpt,  89/502. 

M.  LEPAC'E,  D.  Barnard  and  A.  RUDMIK,  “Optimization  of 
Hierarchical  Directed  Graphs”,  Computer  Languages  6  (1981),  19-34, 
Pergammon  i’ress  Lt-J. 

Z.  Ll  and  P.  Y^W,  ’''''it-li  Jnterproccdural  Analysis  for  Program 
Parallelization  and  .ReMniciuring”,  SIGPLAN  Notices  23,  9 
(September  1988i,  85-99,  ACM  Press.  Proceedings  of  the 

ACM/SIGPLAN  PPEALS  1988,  19-21  July  1988. 


80 


[Mil85] 

[Ney88] 

[Nic85] 

[PHK88] 

[P0I88I 

[RaG69] 

[Sha80| 

lSha78J 

lShi88| 

[SmA881 

lSta89] 

[Stc84] 

[Tar741 

(TjF7()] 


G.  J.  MILNE,  “CIRCAL  and  the  Representation  of  Communication, 
Concurrency  and  Time”,  ACM  Transactions  on  Programming 
Languages  and  Systems  7,  2  (April  1985),  270-298,  ACM. 

A.  D.  NEYRINCK,  “Static  Analysis  of  Aliases  and  Side  Effects  in 
Higher-Order  Languages”,  88-896,  Cornell  University,  Department  of 
Computer  Science,  Ithaca,  NY  14853-7501,  February  1988.  Ph.D. 
Thesis.  January  1988  is  also  given  as  a  date. 

A.  NICOLAU,  “Uniform  Parallelism  Extraction  in  Ordinary  Programs”, 
Proceedings  of  the  ICPP,  1985,  614-618. 

D.  Padua,  W.  L.  Harrison  and  D.  KUCK,  “Machines,  Languages  and 
Compilers  for  Parallel  Symbolic  Computing”  CSRD  729,  Center  for 
Supercomputing  Research  and  Development,  University  of  Illinois, 
September  1988.  (Also  in  Biological  and  Artificial  Intelligence  Systems, 

E.  Clementy  and  S.  Chin,  Eds.  ESCOM,  Leiden,  The  Netherlands,  1988. 
Pp.  453-461.). 

C.  D.  POLYCHRONOPOITOS,  Parallel  Programming  and  Compilers, 
Kluwer  Academic  Publishers,  1988. 

C.  V.  RAMAMOORTHY  and  M.  J.  GONZALEZ,  “A  survey  of  techniques 
for  recognizing  parallel  processable  streams  in  computer  programs”, 
1969  FallJoint  Computer  Conf.  35  (1969),  1-15,  AFIPS  press. 

M.  Sharir,  “Structural  Analysis:  A  New  Approach  To  Flow  Analysis 
in  Optimizing  Compilers”,  Computer  Languages  5  (1980),  141-153, 
F'ergammon  Press  Ltd. 

A.  C.  Shaw,  “Software  Descriptions  with  Flow  Expressions”, 
Transactions  in  Software  Engineering  E-4,  3  (1978),  242-254,  IEEE. 

O.  SHIVERS,  “Control  Flow  Analysis  in  Scheme”,  SIGPLAN  Notices: 
Proceedings  of  the  SIGPLAN'  S8  Conference  on  Programming 
Language  Design  and  Implementation  23,  1  (July  1988),  164-174, 
ACM. 

K.  Smith  and  W.  F.  APPELBE,  “PAT  -  An  Interactive  Fortran 
Parallelizing  Assistant  Tool”,  Proceedings  of  19SS  ICPP  2  (15-19 
August  1988),  58-62,  The  Pennsylvania  State  University  Press. 

R.  M.  Stallman,  Using  and  Porting  GCC,  Free  Software  FouiFatl  'n. 
675  Mass  Ave,  Cambridge,  MA  02139,  1989. 

G.  L.  Steele,  Common  Lisp,  Digital  Press,  1984.  Sti'>'io.  nvj  h.  revised 
edition,  1990. 

R.  E.  Tarjan,  “Testing  Flow'  Graph  Reeucibutiv  '  Tmrnal  of 
Computer  and  System  Sciences  9  (1974),  355-  3v55. 

G.  S.  TJADEN  and  M.  J.  FLYNN,  “Detection  «..r.  ■  -jai  dlel  execution 
independent  instructions”,  IFEF  Transactions  co:  ^ '//YtaLr.r  21,  • 
(October  1970),  889-895. 


81 


[Vee85]  A.  H.  VEEN,  The  Misconstrued  Semicolon,  Technische  Hogeschool 
Eindhoven,  Ph.D.  Thesis,  September  1985. 

[Wad80]  A.  B.  Wadia,  “Generation  of  Node  Lists  Using  Segment  Analysis’’, 
Computer  Languages  5  (1980),  1 15-129,  Pergammon  Press  Ltd. 

[WaG83]  W.  M.  WAITE  and  G.  GOOS,  Compiler  Construction,  Springer-Verlag, 
1983. 

[War84]  J.  WARREN,  “A  Hierarchical  Basis  for  Reordering  Transfomiations’’, 
Conference  Record  of  the  Eleventh  ACM  Symp.  on  Prin.  of  Prog.  Lang., 
Salt  Lake  City,  Utah,  January  1984,  TIl-1%1. 

|Weg831  M-  WEGMAN,  “Summarizing  Graphs  by  Regular  Expressions’’, 
Conference  Record  of  the  Tenth  ACM  Symp.  on  Prin.  of  Prog.  Lang., 
Austin,  Texas,  January  1983,  203-216. 

[Wol89]  M.  WOLFE,  Optunizing  Supercompilers  for  Supercomputers,  The  MIT 
Press,  1989.  Based  on  Wolfe’s  Ph.D.  dissertation,  1982. 

fYuH85]  T.  YUASA  and  M.  HAGIYA,  Kyoto  Common  Lisp  Report,  Research 
Institute  for  Mathematical  Sciences,  Kyoto  University,  1985. 


APPENDIX  A 


A  Catalog  of  Parallelizations 


Here  1  collect  the  rules  of  parallelization  discussed  in  the  text.  First  comes  an 
enumeration  of  the  operations  available  to  the  transformer  in  SPOIL;  then  comes  a 
catalog  of  rules. 

The  operations  available  to  the  transformer  are  expressed  in  S-expression 
notation  (just  because  SPOIL  is  written  in  Lisp).  The  arguments  of  these  fonns  are 
named  either  in  the  fomi  <naiive  or  >name.  In  the  first  case,  it  is  assumed  that  either 
a  constant  is  used  in  the  rule,  or  a  variable  appears  in  that  position,  and  that  variable 
is  expected  to  be  bound  at  the  time  this  part  of  the  guard  is  attempted.  Failure  to  do 
so  causes  the  test  to  fail.  In  the  second  case,  the  name  has  to  be  a  variable  that  has  to 
be  unbound,  or  another  S-expression  containing  unbound  variables.  This  is  the 
mechanism  whereby  tests  and  transfonnations  record  information  for  others  in  the 
same  rule. 

Elementary  tests 

These  tests  query  for  the  correspondence  of  granules,  fragments  and  aspects. 
Typically  they  are  used  early  in  a  rule,  to  gain  access  to  the  material  to  be  tested. 

(:  aspect  <fid  <cciiposition  >variable) 

Given  a  fragment  id  fid  and  a  list  of  fragment  ids  composition,  these  test  binds 
the  variable  to  a  representation  of  the  aspect  of  fid  under  quotient  by 
composition. 

( : graph  <aspect  >variable) 

Given  an  aspect  (as  produced  by  the  : aspect  test),  the  variable  is  bound  to  a 
representation  of  the  graph  that  remains  after  all  the  collapses. 

(:code  <graph  <variable) 

Given  a  graph,  as  produced  by  :graph,  this  test  collects  the  instnictions  into  a 
list  (thereby  mapping  aspects  to  pieces  of  the  original  program). 

( : parse  <graph  <grammar  <variable) 

Given  a  graph  and  a  graph  grammar,  this  test  obtains  the  parse  of  the  graph  with 
respect  to  the  grammar.  In  SPOIL  there  is  only  one  grammar  at  all  times,  so  the 
second  sublomi  is  ignored.  I  have  put  strings  in  the  examples,  as  comments. 
The  <variable  is  u«;ually  a  pattern  that  contains  other 

(:part  <parsevar  Caspect  >fragment-id) 

Given  a  variable  that  presumably  was  involved  in  a  third  subform  of  the 
:parse  test  and  an  aspect,  this  test  succeeds  if  the  subgraph  of  the  fragment 
derived  from  the  parse  variable  coincides  with  one  of  the  fragments,  in  whose 
case  the  third  subfonn  is  bound  to  its  fragment  id.  This  test  appears  in  guards 
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like  ‘if  something  is  a  loop,  and  has  as  its  body  a  decision  between  two  loops’, 
where  just  being  parsable  is  not  all  we  care  about. 

( : execset  <f id  >variable) 

Given  a  fragment  id,  this  binds  the  variable  to  the  S-expression  that  represents 
the  execution  set.  Typically  this  will  be  a  :  choice. 

( :  exists-execset  <f id  <pattem) 

This  is  more  discriminating.  It  tries  to  match  the  current  execution  set  of  fid 
against  the  pattern,  non-deterministically  selecting  a  : choice  that  matches  the 
pattern. 

( :  flow-attribute  <f id  <cor^sition  <attribute  >variable) 

Given  a  fragment  and  a  composition  list  (a  list  of  subffagments  of  it),  this  test 
queries  the  data  flow  information  associated  with  the  attribute.  If  known,  or 
computable,  the  test  succeeds  and  the  variable  is  bound  to  the  value.  This  test 
acts  as  the  interface  with  front  end  in  SPOIL,  as  almost  the  only  thing  SPOIL  ever 
asks  of  the  front  end  is  data  flow  information. 

( : flow-relation  <fidO  <fidl  <fid2  <relation) 

This  is  a  more  specific  query.  The  test  is  whether  the  RELATION  holds 
between  FlDl  and  F1D2  (two  fragment  ids)  when  seen  in  the  context  of  FIDO. 
This  can  only  succeed  or  fail.  Useful  when  checking  whether  specific 
dependence  patterns  occur. 

(:  match  <patteml  <pattem2) 

A  generic  test  that  unifies  both  patterns.  Often  used  to  pick  up  pieces  of  a  result 
from  other  tests. 

( ; intersection  <set  <set  >variable) 

(:  union  <set  <set  >vari.able) 

( : setdiff  <set  <set  >variable) 

( : singleton  <set) 

( : subset  <set  <set) 

(lenptyset  <set) 

( : sameset  <set  <set) 

The  data  flow  information  returned  by  : flow-attribute  is  normally 
expressed  as  a  set  (of  locations,  of  variables,  ...).  The  first  three  succeed  when 
the  specified  set  operation  among  the  given  sets  is  possible  (i.e.,  whenever  the 
first  two  subforms  are  bound  to  constants),  and  then  binds  the  third  subform 
(must  be  a  variable)  to  the  result.  The  rest  don’t  bind  any  variables,  ju.st  succeed 
or  fail  according  to  whether  the  relation  holds. 

( : not  <guard) 

( : and  <guard  . , . ) 

( : or  <guard  . . . ) 

Guards  are  defined  recursively.  These  three  forms  succeed  when  the  logical 
relationship  implied  by  their  names  holds.  The  resulting  bindings  are  as 
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follows:  All  the  bindings  generated  by  all  the  branches  of  :and  are  available; 
all  the  bindings  generated  by  one  of  the  branches  of  :  or  are  available  (which 
branch  is  decided  in  SPOIL  by  choosing  the  first  branch  to  succeed);  and  no 
branches  at  all  from  the  subform  of  :not  are  available. 

(:map  <guarci  cvariablel  <fsetlist  >variable2) 

FSETLIST  must  be  bound  to  a  list  of  fragments.  VARIABLEl  is  bound  to  each 
of  them  in  turn.  For  each  such  binding,  the  GUARD  is  tested.  If  the  GUARD 
succeeds,  the  value  of  VARIABLEl  is  added  to  a  list.  At  the  end,  all  the  values 
of  VARIABLEl  that  satisfied  the  recursive  GUARD  are  in  the  list,  which  is 
then  bound  to  the  VARIABLE2. 

(:  exists  <giiard  <variablel  <fsetlist  >variable2) 

This  is  similar  to  :map,  but  possibly  useful  more  often.  As  soon  as  one  binding 
of  VARIABLEl  in  FSETLIST  satisfies  GUARD,  that  binding  is  given  to 
VARIABLE2  and  the  guard  succeeds. 

(:forall  <giiard  <variable  <fsetlist) 

This  is  similar  to  .'exists.  This  test  fails  if  any  binding  of  VARIABLE  to  the 
elements  of  FSETLIST  does  not  satisfy  the  GUARD.  Else  it  succeeds,  but 
leaves  no  bindings  to  be  observed. 

(:  guard  <guard  <fragment  <fid  <ccnfXDsition  <pgraph) 

This  is  the  recursive  form  of  the  tests. 

(: oracle  <guard) 

This  is  a  back-door  used  when  testing  new  queries  that  haven’t  been 
implemented  yet.  It  just  asks  the  user  to  provide  the  information  that  the 
transformer  needs. 

Transformations 

(:  match  <patteml  <pattem2) 

As  in  the  case  of  guards,  this  is  just  a  generic  unification  pattern  matcher. 

(.■new^code  <codelist  <at-fid  >fid);? 

CODELIST  contains  a  list  of  instructions  (RTL  level),  it  is  either  a  constant  or 
was  produced  via  the  guard  test  :code.  A  new  fragment  is  built  on  that  code, 
the  transformer  binds  the  variable  in  the  third  subform  to  the  new  fragment  id. 
The  second  subform  is  a  pre-existing  fragment  id.  The  new  fragment  is 
considered  a  re-interpretation  of  the  fragment  at  that  position  in  the  hierarchy 
(essentially,  this  means  that  both  fragments  share  their  execution  sets).  This  is 
the  mechanism  used  in  SPOIL  to  perform  reorganization  and  compensation. 

(:new-name  >new-naine) 

This  transformation  generates  a  new  name  (useful  to  rename  variables  or  labels 
in  new  RTL  code). 

( :  with-new-name  <oldnaine  <newnaine  <oldcodelist  >newcodelist) 

Replace  the  NEWNAME  (possibly  generated  by  :new-name,  but  could  also  be 
a  constani  symbol)  for  all  occurrences  of  OLDNAME  in  OLDCODELIST, 
yielding  NEWCODELIST. 
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( :new“execset  <eset  >evariable) 

Given  an  S-expression  representing  an  execution  set,  bind  the  EVARIABLE  to 
it. 

( rccmbine-execsets  <esettag  <esetlist  >eset) 

Given  several  execution  sets  in  a  list  (possibly  created  by  :map),  make  a  new 
one  whose  top  level  operator  is  ESETTAG. 

( :add-to-execset  <fid  <new“choice) 

Replace  the  execution  set  associated  with  FID  with  a  choice  between  the  old 
value  and  the  NEW-CHOICE. 

(••apply  <transfonn  . . .) 

Simply  execute  in  order  all  the  transforms  given  by  the  subforms. 

(:map  <transfonn  <invar  <inlist  <outvar  >outlist) 

Bind  INVAR  to  each  fragment  in  INLIST,  apply  the  TRANSFORM.  If 
OUTVAR  is  bound  by  the  application,  collect  the  value  in  OUTLIST. 

A  Catalog 

Independent  Sequence 
(catalog . add-rule 
cat 

"Independent  Sequence"  ; using  Bernstein's  conditions 

(guard. new  ' (:and 

( : aspect  ?f  ?c  ?a) 

( : graph  ?a  ?pg) 

( :parse  ?pg  "sequence"  ?fparse) 

(imatch  ?^)arse  (:cc  (:cc  ?first  ?tail)  ?exit)) 

(:part  ?first  ?a  ?f.Seq. First) 

(:part  ?tail  ?a  ?f.Seq.Tail) 

(: flow-attribute  ?f.Seq. First  ()  :use  ?Use.f.Seq. First) 
(: flow-attribute  ?f.Seq,Tail  ()  :use  ?Use.f .Seq.Tail) 
(: flow-attribute  ?f.Seq. First  ()  :def  ?Def.f.Seq. First) 
(: flow-attribute  ?f. Seq.Tail  ()  :def  ?Def.f. Seq.Tail) 
(: intersection  ?Def.f.Seq. First  ?Use . f . Seq . Tail  ?1) 

(: intersection  ?Use.f. Seq. First  ?Def . f . Seq . Tail  ?2) 

(: intersection  ?Def.f .Seq. First  ?Def.f .Seq.Tail  ?3) 

( :€snptyset  ?1)  ;not  flow  dependent 

( cesnptyset  ?2)  ;not  anti  dependent 

(renptyset  ?3)  ;not  output  dependent 

)) 

(transform. new  '(:apply 

( :add-to-execset  ?f 
( : parallel 

( : execset  ?f . Seq . First) 

(:execset  ?f .Seq.Tail) )))) ) 
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Forced  Series 
(catalog . add-rule 
cat 

"Forced  Series" 

(guard. new  '  (:and 

( : aspect  ?f  ?c  ?a) 

( :graph  ?a  ?pg) 

( :parse  ?pg  "sequence"  ?fparse) 

( imatch  ?£parse  (:cc  (:cc  ?first  ?tail)  ?exit) ) 

( :part  ?first  ?a  ?f.3eq. First) 

( :part  ?tail  ?a  ?f.Seq.Tail) 

(: flow-attribute  ?f.Seq. First  ()  :use  ?Use.f.Seq. First) 
(: flow-attribute  ?f.Seq.Tail  ()  ruse  ?Use.f .Seq.Tail) 
(: flow-attribute  ?f.Seq. First  ()  rdef  ?E)ef.f.Seq. First) 
( :flow-at tribute  ?f. Seq.Tail  ()  rdef  ?E)ef.f  .Seq.Tail) 

( r intersection  ?E)ef.f.Seq. First  ?Use.f  .Seq.Tail  ?1) 

( r intersection  ?Use.f.Seq. First  ?E)ef.f  .Seq.Tail  ?2) 

( r  intersection  ?Def.f.Seq. First  ?E)ef.f  .Seq.Tail  ?3) 

( ror 

(rnot  (renptyset  ?!))  ;flow  dependent 
(rnot  (renptyset  ?2))  ;anti  dependent 
(rnot  (renptyset  ?3))  /output  dependent 
))) 

(tiransform.new  '  (r apply 

( radd-to-execset  ?f 
( r  series 

( r  execset  ?f . Seq . First ) 

(rexecset  ?f .Seq.Tail) )))) ) 

Loop  Recognition 
(catalog . add-rule 
cat 

"Loop  Recognition" 

(guard. new  '  (rand 

(r aspect  ?f  ?c  ?a) 

(r  match  ?c  (?IioqpBody) ) 

( r graph  ?a  ?fpg) 

( rparse  ?^pg  "loop  ?fparse) 

( rmatch  ?^parse  (rcc  (rloop  ?Body)  ?Exit) ) ) ) 

(transform . new  '(rapply 

(radd-to-execset  ?f 
( r sloop  ( rexecset  ?LoopBody) ) ) ) ) ) 

Loop  Distribution 
(catalog . add-rule 
cat 
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"Loop  Distribution" 

(gioard.new  '  (:and 

( :ex6cset  ?f  ?e) 

(:match  ?e  (:sloc^  (:parallel  .  ?parts))))) 
(transform. new  '  (: apply 

(cmap  ( rnew-exGcset  (: sloop  ?part)  ?newlopp) 
?part  ?parts 
?newlopp  ?newloqps) 

( : combine-execsets  : parallel  ?newloqps  ?neweset) 
( :add-to-execset  ?f  ?neweset) ) ) ) 


DO  to  DOALL 

(catalog,  add-rule  ;l-loc^s/swap 

cat 

"SLoop  ->  PLoop  (DO  ->  DOALL)" 

(guard. new  ' (;and 

( rexists-execset  ?f  (tsloc^  ?body) ) 

(:  match  ?body  (rexecset  ?fb)) 

( :  flow-relation  ?f  ?fb  ?fb  :  loop- independent) ) ) 
(transform. new  '  (: apply 

( :new-execset  (:ploop  ?body)  ?newloc^) 

( : add-to-execset  ?f  ?newloop) ) ) ) 


Partial  Distribution 

(catalog. add-rule  /polyc/loqp3 

cat 

"(SLOOP  X  .  REST)  ->  (SERIES  (PLOOP  X)  (SLOOP  REST))" 

(guard. new  '  (:and 

(rexists-execset  ?f  (rslocp  ?x  .  ?rest) ) 

(rmatch  ?x  (rexecset  ?fx) ) 

( :  flow-relation  ?f  ?fx  ?fx  :  loop- independent) ) ) 
(transform. new  '  (: apply 

( : new-execset  ( : series 

(rplocp  ?x) 

(: sloop  .  ?rest) )  ?newloop) 

( ; add-to-execset  ?f  ?newloop) ) ) ) 

( catalog . add-ru le  ; poly c /loop2 

cat 

"(SLOOP  REST  X)  ->  (SERIES  (SLOOP  REST)  (PLOOP  X))" 
(giiard.new  '  (rand 

(rexists-execset  ?f  (r sloop  ?rest  ?x) ) 

(rmatch  ?x  (rexecset  ?fx) ) 

( r  flow-relation  ?f  ?fx  ?fx  r  locp- independent) ) ) 
(transform . new  ' (r apply 

( r new-execset  (r series 
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(: sloop  ?rest) 

( :ploqp  ?x) )  ?newloop) 
( : add-to-execset  ?f  ?newloop) ) ) ) 


DOACROSS 

(catalog. add-rule  ;wolfe/doacr 

cat 

"DQACRaSS" 

(guard. new  '  (:and 

( :  exists-execset  ?f  (:  sloop  .  ?loqpt)ody) ) 

(: match  ?c  (?body))  ;only  vdien  the  entire  body  is  visible 
( :  exists-execset  ?body  ( :  series  .  ?loopibody) ) 

(: flow-attribute  ?f  (?body)  : synchronized-code  ?synch-code) ) ) 
(transform. new  '  (: apply 

(:map  (:  apply 

( :match  ?synch  (?fid  ?newcode) ) 

( :new-code  ?newcode  ?fid  ?nfid) 

( :new-execset  (:execset  ?nfid)  ?enfid) ) 

?synch  ?synch-code  ?enfid  ?enfids) 

( : combine-execsets  :ploop  ?enfids  ?new-eset) 

(: add-to-execset  ?f  ?new-eset) ) ) ) 

Ignorable  Self  Anti-dependences 

(catalog. add-rule  ;wolfe  p.64 

cat 

"Ignorable  self-antidependence  cycles" 

(guard. new  ' (:and 

(: exists-execset  ?f  (;sloqp  (rexecset  ?fb) ) ) 

(: flow-relation  ?f  ?fb  ?fb  : flow-dependent) 

(:rot  (:  flow- relation  ?f  ?fb  ?fb  :  anti -dependent ) ) 

(:not  (:  flow- relation  ?f  ?fb  ?fb  :  output -dependent ) ) 

(: flow-attribute  ?fb  ()  ?def-locs  ?d) 

( : match  ?d  (?target) ) 

( : code  ?fb  ?oldcode) ) ) 

(transform. new  ' (: apply 

(:new-name  ?new-array) 

( : new-code  ( (call-insn  ' copy  ?target  ?new-array) ) 
nil  ?cqpy-in) 

( :with-new-name  Ttarget  ?new-array  ?oldcode  ?newcode) 

(:  new-code  ?newcode  ?nevdx>dy) 

(: new-code  ((call-insn  'copy  ?new-array  Ttarget)) 
nil  ?cqpy-out) 

( : new-execset  ( : series 
?copy-in 

(;ploqp  Tnewbody) 

?ccpy-out) 


89 


?new-eset) 

( :add-to-execset  ?f  ?new-eset) ) ) ) 


APPENDIX  B 


Example  Programs 


Test  Suite 

A  number  of  programs  were  used  during  the  testing  phase  of  SPOIL.  They  are 
collected  here  for  ease  of  reference. 

minmax 

This  program  was  originally  the  synthetic  example  I  used  to  provide  details  on  my 
work.  It  contains  mostly  mid-to-coarse  level  parallelism,  in  the  form  of  independent 
reductions  of  minimum  and  maximum.  The  same  loci  of  parallelization  are  open,  of 
course,  to  “telescopic”  reductions,  where  oidy  a  logarithmic  number  of  iterations  is 
needed.  In  its  simplicity,  it  encodes  most  of  the  typical  characteristics  of  promising 
parallelizable  code. 

/* 

-  minmax (int  x[],  int  n,  int  *min,  int  *max) 

* 

*  X  has  indices  0.  .N.  At  the  end,  the  minimum  and  the  maximum  in  X  are 

*  returned  through  the  MIN  and  MAX  pointers. 

* 

*  $Id:  minmax. c,v  1.1  89/09/25  10:28:38  quiroz  Ejqp  Locker:  quiroz  $ 

*/ 

void 

minmax(int  x[],  int  n,  int  *min,  int  *max) 

{ 

int  i;  /*  loop  index  */ 

int  localmin,  localmax;  /*  temporaries  */ 

localmin  =  localmax  =  x[0] ; 

i  =  1; 

vdiile  (  i  <=  n  )  { 
if  (x[i]  <  localmin) 
localmin  =  x[i]  ; 
if  (x[i]  >  localmax) 
localmax  =  x[i]  ; 
i  +=  1; 

} 
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*min  =  localmin; 

*max  =  localmax; 
return; 

} 

minmaxsum 

This  is  just  the  ininmax  program,  wiih  a  little  extra  activity  to  calculate  also  the  sum. 
Its  reason  for  being  was  to  provide  a  data  point  of  a  different  size  when  the  cost  of 
fragment  and  aspect  construction  was  still  an  issue. 

/* 

-  minmaxsum (int  x[],  int  n,  int  *min,  int  *max,  int  *sum) 

* 

*  X  has  indices  0.  .N.  At  the  end,  the  minimum,  the  maximum,  and  the 

*  sum  of  X  are  returned  tlirough  the  MIN,  MAX,  and  SUM  pointers. 

* 

*  $Id:  minmax.c,v  1.1  89/09/25  10:28:38  quiroz  E:q>  Locker:  quiroz  $ 
*/ 

void 

mininaxsxjm(int  x[],  int  n,  int  *min,  int  *max,  int  *st3m) 

{ 

int  i;  /*  loop  index  */ 

int  localmin,  locallmax;  /*  tenporaries  */ 

int  localsum; 

localmin  =  localmax  =  XtU]; 
localsum  =  0; 

i  =  1; 

vrtiile  (  i  <=  n  )  { 
if  (x[i]  <  localmin) 
localmin  =  x[i]; 
if  (x[i]  >  localmax) 
localmax  =  x[i] ; 
if  (x[i]  >  0) 

localsum  +=  x[i] ;  /*  hnmm.  .  .  */ 

i  4=  1; 

} 

*min  =  localmin; 

*max  =  localmax; 

*sum  =  localsum; 
return; 


} 
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loopl 

The  following  three  programs  come,  essentially,  from  [P0I88,  sec.  2.3.1,  p.  22]. 

/*  2.3.1,  p.22  up  */ 

/*  SI  and  S2  can  be  vectorized  independently  */ 
void 

oneloop(a,  b,  c,  d,  n,  k) 

double  *a,  *b,  *c,  *d; 

int  n; 

double  k; 

{ 

int  i; 

for  (i  =0;  i  <  n;  i++)  { 
a[i]  =  b[i]  +  c[i]  ; 
d[i]  =  b[i]  *  k; 

} 

} 

loop2 

/*  2.3.1,  p.22  dovm  */ 

/*  S3  can  be  taken  out  of  the  loop  and  vectorized  : 

(sloop  H  .  Rest)  ->  (series  (ploop  H)  (sloop  Rest) ) 

(sloop  Rest  I  (T) )  ->  (series  (sloop  Rest)  (sloop  T) ) 

where  H  (or  T)  don't  carry  any  dependencies. 

*/ 

void 

loop  (a,  b,  c,  n,  k) 

doi±>le  *a,  *b,  *c; 

int  n; 

doi±>le  k; 

{ 

int  i; 

for  (i  =0;  i  <  n;  i++)  { 
a[i+l]  =  b[i-l]  +  c[i]; 
b[i]  =  a[i]  *  k; 
c[i]  =  b[i]  -  1; 

} 

} 


/*  SI  */ 
/*  S2  */ 
/*  S3  */ 


/*  SI  */ 
/*  S2  */ 
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loop3 

/*  2.3.1,  down--MX)IFIED  */ 

/*  S3  can  be  taken  out  of  the  loop  and  vectorized  : 

(sloop  H  .  Rest)  ->  (series  (ploqp  H)  (sloop  Rest) ) 
(sloop  Rest  I  (T) )  ->  (series  (sloop  Rest)  (sloop  T) ) 

vdiere  H  (or  T)  don't  carry  any  dependencies. 

*/ 

void 

loop  (a,  b,  c,  n,  k) 


double 

*a,  *b,  *c; 

int 

n; 

double 

k; 

int 

i; 

for  (i  =  0; 

i  <  n;  i-H-)  { 

c[i] 

=  b[i]  -  1; 

/* 

S3 

*/ 

a[i+l] 

=  b[i-l]  +  c[i]/ 

/* 

SI 

*/ 

b[i] 

=  a[i]  *  k; 

/* 

S2 

*/ 

} 


} 

wolfe-4.2 

This  program  is  a  translation  of  the  example  given  in  [Wol89,  sec.  4.2,  p.  74J. 

/* 

*  Adapted  fron  Wolfe's  C^imizing  Supercarpilers  for  Superconputers,  p.  74 


#define  N  100 
void 

exarrple(int  a[N]  [N] ,  int  b[N]  [N]) 

{ 

int  i,  j; 

/*  Summary: 

* 

*  SI  data(<,<)  S2 

*  SI  anti(=,=)  S2 

* 

*  So  the  inner  loop  doesn't  carry  the  antidependence,  and  the  inner  looj 
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*  doesn't  see  it  when  executed  serially. 

★ 


*/ 


for  (i  =  2;  i  <  N;  i++)  /*  serial  loop  */ 

for  (j  =  2;  j  <  N;  j++)  {  /*  parallel  loop  */ 

/*S1*/  a[i][j]  =b[i][j]  +  2; 

/*S2*/  b[i] [j]  =  a[i-l] [j-1]  -  b[i] [j] ; 

} 

} 

doacr 

This  program  comes  from  [Wol89,  sec.  4.3,  p.  76]. 

/*  Vfolfe,  p.  76  */ 


void 

doacr  (a,  b,  c,  d,  e,  n) 


int 

n; 

double 

*a 

int 

i; 

*b. 


*c. 


M,  *e; 


(i  =  1;  i  < 

n;  i++)  { 

/*  wait (S3, 

I-l)  */ 

a[i]  =  b[i] 

+  c[i-l] ; 

/* 

SI 

*/ 

/*  post (SI, 

I)  */ 

d[i]  =  a[i] 

*  2.0; 

/* 

S2 

*/ 

/*  wait (SI, 

I-l)  */ 

c[i]  =  a[i-l]  +  c[i] ; 

/* 

S3 

*/ 

/*  post (S3, 

I)  */ 

/*  wait (S3, 

1-2)  */ 

e[i]  =  d[i] 

+  c[i-2]; 

/* 

S4 

*/ 

} 

} 


decomp 

The  next  two  programs  fonn  a  linear  system  solver,  taken  from  the  NETLIB  archive 
The  original  (FORTRAN)  versions  are  from  George  Forsythe,  Mike  Malcolm,  and 
Cieve  Moler.  Those  FORTRAN  versions  were  translated  by  the  Fortran  to  C  service 
also  available  at  NETLIB.  For  more  information  on  NETLIB,  send  electronic  mail  to 
netlib@research.att.ccm.  A  message  body  that  reads  simply: 

send  index 

will  provide  the  necessary  information. 
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/*  —  translated  by  f2c  (version  of  27  April  1990  12:27:33)  . 

You  must  link  the  resulting  object  file  with  the  libraries: 
-1F77  -1177  -Im  -Ic  (in  that  order) 

*/ 

#include  "f2c.h" 

/*  Subroutine  */  int  decatp_(ndim,  n,  a,  cond,  ipvt,  work) 

integer  *ndim,  *n; 

doublereal  *a,  *cond; 

integer  *ipvt; 

doublereal  *work; 

{ 

/*  System  generated  locals  */ 
integer  a_diinl,  a_cffset,  i_l,  i_2,  i_3; 
doublereal  d_l,  d_2; 

/*  Local  variables  */ 

static  integer  i,  j,  k,  m; 

static  doublereal  t,  anorm; 

extern  /*  Subroutine  */  int  solve_(); 

static  doublereal  ynorm,  znorm; 

static  integer  kb; 

static  doublereal  ek; 

static  integer  kttil,  nml,  kpl; 


/*  decomposes  a  doiisle  precision  matrix  by  gaussian  elimination  */ 
/*  and  estimates  the  condition  of  the  matrix.  */ 

/*  use  solve  to  conpute  solutions  to  linear  systems.  */ 

/*  input . .  */ 

/*  ndim  =  declared  row  dimension  of  the  array  containing  a.  */ 

/*  n  =  order  of  the  matrix.  */ 

/*  a  =  matrix  to  be  triangularized.  */ 

/*  output . .  */ 

/*  a  contains  an  upper  triangular  matrix  u  and  a  permuted  */ 

/*  version  of  a  lower  triangular  matrix  i-1  so  that  */ 

/*  (permutation  matrix)  *a  =  l*u  .  */ 
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/*  cond  =  am  estimate  of  the  condition  of  a  .  */ 

/*  for  the  linear  system  a*x  =  b,  changes  in  a  amd  b  */ 

/*  may  cause  chang-  s  cond  times  as  large  in  x  .  */ 

/*  if  cond+1.0  .eq.  cond  ,  a  is  singular  to  working  */ 

/*  precision,  cond  is  set  to  l.Od+32  if  exact  */ 

/*  singularity  is  detected.  */ 

/*  ipvt  =  the  r>ivot  vector.  */ 

/*  ipvt(k'.  the  index  of  the  k-th  pivot  row  */ 

/*  ipvt(n)  •  (-1) (ni. -ber  of  interchanges)  */ 

/*  work  space. .  the  vector  work  must  be  declared  and  included  */ 
/*  in  the  call,  its  input  contents  are  ignored.  */ 

/*  its  output  contents  are  usually  unirrportant .  */ 

/*  the  determinant  of  a  can  be  obtained  on  output  by  */ 

/*  det(a)  =  ipvt(n)  *  a(l,l)  *  a(2,2)  *  ...  *  a(n,n)  .  */ 


/*  Pairameter  adjustments  */ 

— work; 

— ipvt; 

a_diml  =  *ndim; 
a_offset  =  a_diml  +  1; 
a  -=  a_offset; 

/*  Function  Body  */ 
ipvt[*n]  =  1; 
if  (*n  ==  1)  { 
goto  L80; 

} 

nml  =  *n  -  1; 

/*  cocrpute  1-norm  of  a  */ 

amorm  =  0 . ; 
i_l  =  *n; 

for  (j  =  1;  j  <=  i_l;  ++j)  { 
t  =  0.; 
i_2  =  *n; 

for  (i  =  1;  i  <=  i_2;  ++i)  { 

t  4=  (d_l  =  a[i  +  j  *  a_diml] ,  abs(d_l)); 

/*  L5:  */ 

} 

if  (t  >  anorm)  { 
anorm  =  t; 
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} 

/*  LIO:  */ 

} 


/*  gaussian  elimination  with  partial  pivoting  */ 
i_l  =  nml; 

for  (k  =  1;  k  <=  i_l;  -H-k)  { 
kpl  =  k  +  1; 

/*  find  pivot  */ 

m  =  k; 
i_2  =  *n; 

for  (i  =  kpl;  i  <=  i_2;  ++i)  { 

if  ( (d_l  =  a[i  +  k  *  a_diinl],  abs(d_l))  >  (d_: 
,  abs(d_2)))  { 
m  =  i; 

} 

/*  L15:  */ 

} 

ipvt[k]  =  m; 
if  (m  !=  k)  { 

ipvt[*n]  =  -ipvt[*n]; 

} 

t  =  a[m  +  k  *  ajdiml] ; 

a[m  +  k  *  ajdiml]  =  a[k  +  k  *  ajdiml]; 

a[k  +  k  *  ajdiml]  =  t; 

/*  skip  step  if  pivot  is  zero  */ 

if  (t  =  0.)  { 
goto  L35; 

} 

/*  ccrpute  multipliers  */ 

i_2  =  *n; 

for  (i  =  kpl;  i  <=  i_2;  -f+i)  { 

a[i  +  k  *  a_diml]  =  -a[i  +  k  *  a_diml]  /  t; 

/*  L20:  */ 

} 


/* 


interchange  and  eliminate  by  columns  */ 


=  a[m  +  k  *  ajdiml] 


i  2  =  *n; 
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for  (j  =  Jqpl;  j  <=  i_2;  44 j)  { 
t  =  a[m  4  j  *  a_diml]  ; 
a[m  4  j  *  ajdiml]  =  a[k  4  j  *  ajdiml]  ; 
a[k  4  j  *  ajdiml]  =  t; 
if  (t  =  0.)  { 
goto  L30; 

} 

i_3  =  *n/ 

for  (i  =  Jq>l;  i  <=  i_3;  44i)  { 
a[i  4  j  *  ajdiml]  4=  a[i  4  k  *  ajdiml]  *  t; 

/*  L25:  */ 

} 

L30: 

t 

} 

L35: 

t 

} 

/*  cond  =  (1-norm  of  a)* (an  estimate  of  1-norm  of  a-inverse)  */ 
/*  estinitite  obtained  by  one  step  of  inverse  iteration  for  the  */ 
/*  small  singular  vector,  this  involves  solving  two  systems  */ 

/*  of  equations,  (a-transpose)  *y  =  e  and  a*z  =  y  where  e  */ 

/*  is  a  vector  of  4l  or  -1  chosen  to  cause  growth  in  y.  */ 

/*  estimate  =  (1-norm  of  z)/ (1-norm  of  y)  */ 

/*  solve  (a-transpose) *y  =  e  */ 

i_3  =  *n; 

for  (k  =  1;  k  <=  i_l;  44k)  { 
t  =  0.; 
if  (k  =  1)  { 
goto  L45; 

} 

kml  =  k  -  1; 
i_2  =  kml; 

for  (i  =  1;  i  <=  i_2;  44i)  { 

t  4=  a[i  4  k  *  ajdiml]  *  work[i]; 

/*  L40:  */ 

} 

L45: 

ek  =  1 .  ; 
if  (t  <  0.)  { 
ek  =  -1 . ; 

} 

if  (a[k  4  k  *  a  diml]  =0.)  { 
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goto  L90; 

} 

work[k]  =  -(ek  +  t)  /  a[k  +  k  *  ajdintil]; 
/*  L50:  */ 

} 

i_l  =  nml; 

for  (kb  =1;  kb  <=  i_l;  -H-kb)  { 
k  =  *n  -  kb; 
t  =  0 .  ; 

=  k  +  1; 
i_2  =  *n; 

for  (i  =  kpl;  i  <=  i_2;  ++i)  { 

t  +=  a[i  +  k  *  a_diml]  *  work[i]; 

/*  L55:  */ 

} 

work [k]  =  t  +  work [k] ; 
m  =  ipvt [k] ; 
if  (m  =  k)  { 
goto  L60; 

} 

t  =  work [m] ; 
work[m]  =work[k]; 
work[k]  =  t; 

L60: 


} 

ynorm  =  0 . ; 
i_l  =  *n; 

for  (i  =  1;  i  <=  i_l;  -H-i)  { 
ynorm  +=  (ci_l  =  work[i],  abs(ci_l)); 
/*  L65:  */ 

} 


/*  solve  a*z  =  y  */ 

solve_(ndim,  n,  &a  [a_offset] ,  Swork  [1] ,  &  ipvt  [1] )  ; 

znorm  =  0 . ; 
i_l  =  *n; 

for  (i  =  1;  i  <=  i_l;  -H-i)  { 
znorm  +=  (d_l  =  work[i],  abs(d_l)); 

/*  L70:  */ 

} 


/* 


estimate  condition  */ 
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*cond  =  anorm  *  znorm  /  ynom; 
if  (*cond  <  1.)  { 

*cond  =  1 . ; 

} 

return  0; 

/*  1-by-l  */ 

L80: 

*a)nd  =  1 . ; 

if  (a[a_diml  +  1]  !=  0.)  { 
return  0; 

} 


/*  exact  singularity  */ 

L90: 

*cond  =  le32; 
return  0; 

}  /*  deccrp_  */ 

#ifdef  uNdEfInEki 

comments  from  the  converter:  (stderr  from  f2c) 
deconp: 

Warning  on  line  168:  local  variable  dsign  never  used 
#endif 

solve 

/*  —  translated  by  f2c  (version  of  27  i^ril  1990  12:27:33) 

You  must  link  the  resulting  object  file  with  the  libraries 
-1F77  -1177  -Im  -Ic  (in  that  order) 

*/ 

#include  "f2c.h" 

/*  Subroutine  */  int  solve_(ndim,  n,  a,  b,  ipvt) 
integer  *ndim,  *n; 
doublereal  *a,  *b; 
integer  *ipvt; 

{ 

/*  System  generated  locals  */ 
integer  a_diml,  a_offset,  i_l,  i_2; 

/*  Local  variables  */ 
static  integer  i,  k,  m; 
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static  doi±)lereal  t; 

static  integer  kb,  kitil,  nml,  kpl; 


/*  solution  of  linear  system,  a*x  =  b  .  */ 

/*  do  not  use  if  deccrp  has  detected  singularity.  */ 

/*  input . .  */ 

/*  ndim  =  declared  row  dimension  of  array  containing  a  .  */ 
/*  n  =  order  of  matrix.  */ 

/*  a  =  triangularized  matrix  obtained  from  deccnp  .  */ 

/*  b  =  right  hand  side  vector.  */ 

/*  ipvt  =  pivot  vector  obtained  from  decaip  .  */ 

/*  output . .  */ 

/*  b  =  solution  vector,  x  .  */ 


/*  forward  elimination  */ 

/*  Parameter  adjustments  */ 
--ipvt; 

— b; 

a_diml  =  *ndim; 
a_offset  =  a_diml  +  1; 
a  -=  a_offset; 

/*  Function  Body  */ 
if  (*n  =  1)  { 
goto  L50; 

} 

nml  =  *n  -  1; 
i_l  =  nml; 

for  (k  =  1;  k  <=  i_l;  ++k)  { 
kpl  =  k  +  1; 
m  =  ipvt  [k]  ; 
t  =  b[m]; 
b[m]  =  b[k]; 
b[k]  =  t; 
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i_2  =  *n; 

for  (i  =  kpl;  i  <=  i_2;  -f+i)  { 

b[i]  +=  a[i  +  k  *  a_diml]  *  t; 
/*  LIO:  */ 

} 

/*  L20:  */ 

} 


/*  back  substitution  */ 
i_l  =  mnl; 

for  (kb  =1;  kb  <=  i_l;  ++kb)  { 
kml  =  *n  -  kb; 
k  =  kml  +1; 

t>[k]  /=  a[k  +  k  *  a_diml]; 
t  =  -b[k]; 
i_2  =  kml; 

for  (i  =  1;  i  <=  i_2;  ++i)  { 

b[i]  +=  a[i  +  k  *  a_diml]  *  t; 

/*  L30:  */ 

} 

/*  L40:  */ 

} 

L50: 

b[l]  /=  a[a_diml  +  1]  ; 
return  0; 

}  /*  solve_  */ 

#ifdef  uNdEfInEd 

ccmments  frcm  the  converter:  (stderr  fran  f2c) 
solve: 

#endif 

frac 

/*  The  following  value?  are  used  to  test  for  too  small 
and  too  large  values  of  v  for  an  integer  fraction  to 
represent . 

These  values  are  currently  set  fc  th'  mdal  5890. 

*/ 

#define  MAX  4.0e+10 
#define  MIN  3.0e-10 


double 
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frac(v,  n,  d,  error) 
dotible  V,  error; 
int  *n,  *d; 

{ 

/* 

given  a  nxunber,  v,  this  function  outputs  two  integers, 
d  and  n,  such  that 

V  =  n  /  d 

to  accuracy 

^silon  =  I  (v  -  n/d)  /  v  |  <=  error 

input:  V  =  decimal  ntniber  you  want  replaced  by  fraction, 
error  =  accuracy  to  vdiich  the  fraction  should 
represent  the  number  v. 

output:  n  =  numerator  of  representing  fraction, 
d  =  denominator  of  representing  fraction. 

return  value:  -1.0  if  (v  <  MIN  |  |  v  >  MAX  |  |  error  <  0.0) 
I  (v  -  n/d)  /  V  I  otherwise. 

Note:  This  program  only  works  for  positive  nimbers,  v. 

reference:  Jerome  Spanier  and  Keith  B.  Oldham,  "An  Atlas 
of  Functions,"  Springer-Verlag,  1987,  pp.  665-7. 

*/ 

int  D,  N,  t; 

double  epsilon,  r,  m,  fabs(); 


if  (v  <  MIN  I  I  V  >  MAX  I  I  error  <  0.0) 
return (-1.0)  ; 

*d  =  D  =  1; 

*n  =  (int)v; 

N  =  (*n)  +  1; 
goto  three; 

one:  if  (r  >  1.0) 

goto  two; 
r  =  1.0/r; 

two:  N  4=  (*n)*(int)r; 

D  +=  (*d)*(int)r; 

(*n)  +=  N; 
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(*d)  +=  D; 

three:  r  =  0.0; 

if  (v*(*d)  ==  (doiJble)  (*n)) 
goto  four; 

r  =  (N  -  v*D)/(v*(*d)  -  (*n)); 
if  (r  >  1.0) 

goto  four; 

t  =  N; 

N  =  (*n) ; 

*n  =  t; 
t  =  D; 

D  =  (*d)  ; 

*d  =  t; 

four:  printf  ("%d/%d",  *n,  *d)  ; 

epsilon  =  fabs  (1.0  -  (*n)  /  (v*  (*d) ) )  ; 
if  (epsilon  <=  error) 
goto  six; 
m  =  1.0; 
do  { 

m  *=  10.0; 

}  v^ile  (m*epsilon  <  1.0); 
epsilon  =  1.0/m  *  ( (int) (0.5  +  m*epsilon) ) ; 
six:  printf ("  epsilon  =  %q0,  epsilon); 
if  (^silon  <=  error) 
return  (epsilon) ; 
if  (r  !=  0.0) 

goto  one; 

} 


sll5 

The  five  following  programs  come  also  from  NETLIB,  and  are  extracts  from  the  suite 
of  vectorization  tests  vec±or.f,  converted  from  FORTRAN  to  C  by  hand.  The 
translation  does  not  account  for  the  offsets  needed  to  convert  FORTRAN  1 -based 
indices  into  C  0-based  indices,  so  the  codes  shown  are  not  strictly  equivalent  to  the 
originals,  but  show  the  same  specific  problems  in  a  cleaner  setting. 

void 

sll5(do\3ble  **aa,  int  n) 

{ 

int  i,  j,  k; 

for  (j  =  1;  j  <  n;  j++)  /*  do  320  j  */ 

for  (k  =  1;  k  <  j-1;  k++)  /*  do  320  k  */ 

for  (i  =  k+1;  i  <  n;  i++)  /*  do  320  i  */ 

aa[i][j]  =aa[i3[j]  +  aa[i]  [k]  *  aa[k][j]; 
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} 


sl23 


void 

sl23  (doxable  *a, 
{ 

int 

int 


doi±)le  *b,  doubio  *c,  int  n) 

j  =  0; 
i; 


for  (i  =1;  i  <  n;  i++)  {  /*  do  50  i  */ 

j  =  j  +  1; 
a[j]  =  b[i]  ; 

if  (c[i]  >  0)  { 

j  =  j  +  1; 
a[j]  =  c[i]; 

} 

} 

} 


sl24 

void 

sl24  (double  *a,  double  *b,  double  *c,  int  n) 

{ 

int  j  =  0; 

int  i; 

for  (i  =1;  i  <  n;  i++)  {  /*  do  60  i  */ 

if  (b[i]  >  0)  { 

j  =  j  +  1; 
a[j]  =  b[i]; 

}  else  { 

j  =  j  +  1; 
a[j]  =  c[i]  ; 

} 


} 

} 

sl31 

void 

sl31  (doxable  *a,  doxjble  *b,  int  n) 

{ 

int  m  =  1; 


106 


int  i; 

if  (a[l]  >  0) 
a[l]  =  b[l]; 

for  (i  =2;  i  <  n-1;  i++)  /*  do  340  i  */ 

a[i]  =  a[i-hn]  +  b[i]; 

} 

S2710 

void 

s2710  (doiable  *a,  doubxe  *b,  dovible  *c,  doiable  x,  int  n) 

{ 

int  i; 

for  (i  =1;  i  <  n;  i++)  /*  do  790  i  */ 

if  (a[i]  >b[i])  { 

a[i]  =  a[i]  -  b[i]; 

if  (n  >  10) 

c[i]  =  fabs(c[i]) ; 

else 

cti]  =  0.0/ 


}  else  { 

b[i]  =  a[i]/ 

if  (X  >  0) 

c[i]  =  a[i]  ; 

else 


pat.f 

This  and  the  next  program  are  from  [SmA88].  The  FORTRAN  version  is  described 
in  the  paper  of  the  reference. 

c  Fran  [ .  PAT  icpp  1988  { ,  page  62 } .  ] 


real  a (20),  b(20),  sum,  max 
integer  i,  j 
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call  init 

Slim  =  C 
j  =  20 

do  100  i  =  1;20 
sum  -  sum  +  a(i) 

a(i)  =  b(j)  +  a(i  -  1)  !  this  seems  like  an  error  for  i: 

if  (b(i)  .gt.max)  max  =  b(i)  !  what  does  this  do? 

j  =  j  -  1 

100  continue 

print  105,  sum,  max 

105  format ('anm  :  ’ ,  f8.5,  '  max  :  f8.5) 

end 

pat.c 

This  is  pretty  much  a  direct  translation  (again,  disregarding  the  difference  in  index 
origin)  of  the  pat.f  program.  The  most  important  difference  is  that  the  original 
contains  an  illegal  subscript:  when  i  is  1,  the  original  makes  a  reference  to  a(0), 
which  is  presumably  illegal  FORTRAN.  The  C  version  adds  a  conditional,  which 
introduces  a  little  more  structure  for  the  parallelizer  to  work  on. 

/*  Adapted  from  Smith  and  J^ipelbe,  ICX3*  1988, 

*  ' 'PAT  —  An  Interactive  Fortran  Parallelizing  Assistant  Tool' ' 

*/ 

int 
main() 

{ 

float 
int 

extern  void 
init  0  ; 
sum  =  0/ 

j  =  20; 

for  (i  =  0;  i  <  20;  i-H-)  { 
sum  =  sum  +  a[i] ; 
if  (i  >  0) 

a[i]  =  b[j]  +  a[i  -  1]  ; 
if  (b[i]  >  max) 
max  =  b[i]  ; 

j  =  j  -  1; 


a  [20],  b[20],  sum,  max; 

i.  j; 

init  0  ; 
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printf ("sum  :  %f8.5  max  :  %f8.50,  sum,  max); 

} 

Steele 

This  program  was  contributed  (in  its  FORTRAN  form)  by  Dr.  Guy  Steele,  from 
Thinking  Machines  Corporation  (personal  communication,  12  December  1988).  It 
shows  a  variety  of  loci  for  parallelization. 

/* 

*  Adapted  from  Guy  Steele,  personal  conmunication,  Dec.  1988 
*/ 

# include  <stdio.h> 

# include  <tnath.h> 

#define  SQR(x)  ( (x) * (x) ) 

void 

niigrt(p0,  p,  v,  nt,  nx,  nz,  dt,  dx,  pp,  itinc) 
int  nt,  nx,  nz; 

do\±)le  dt,  dx; 

int  itinc; 

do\±)le  pO  [nt]  [nx] ; 

double  p[nz] [nx] ; 

double  v[nz][nx]; 

dox±ile  pp[nz]  [nx]  [3] ; 

{ 

int  ix,  iz,  it,  i;  /*  loop  indices  */ 

int  itO,  itl,  it2,  it3;  /*  auxiliaries  */ 

doiable  ddx,  ddz;  /*  more  auxiliaries  */ 

for  (ix  =  1;  ix  <  rrx;  ix-H-) 

for  (iz  =1;  iz  <  nz;  iz++)  { 

v[iz]  [ix]  =  SQR(v[iz]  [ix]/2  *  dt/dx)  ; 
if  (v[iz] [ix]  >  0.5) { 

(void) fpr int f (stderr,  "E3U^R:  v*dt/dx  =  %g  >  0.50, 
sqrt(v[iz] [ix])) ; 

exit (1) ; 

} 

}; 

for  (i  =  1;  i  <  3;  .i++)  /*  parallelizable  */ 

for  (ix  =1;  ix  <  nx;  ix++) 

for  (iz  =  1;  iz  <  nz;  iz-H-) 

FP[iz] [ix] [i]  =  0.0; 


itl  =  1; 
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it2  =  2; 
it3  =  3; 

for  (it  =  1;  it  <  nt;  it++)  { 
itl  =  it3; 
itO  =  itl; 
itl  =  it2; 
it2  =  it3; 

for  (ix  =  1;  ix  <  nx;  ix++) 

Fp[l]  [ix]  [it2]  =  pO[nt-it+l]  [ix]  ; 

for  (ix  =  2;  ix  <  nx  -  1;  ix++)  /*  parallelizable  */ 
for  (iz  =  2;  iz  <  nz  -  1;  iz++)  { 
ddx  =  (pp[iz] [ix+1] [it2] 

-  2  *  pp[iz] [ix] [it2] 

+  pp[iz] [ix-1] [it2])  ; 

ddz  =  (pp[iz+l] [ix] [it2] 

-  2*pp[iz] [ix] [it2] 

+  pp[iz-l] [ix] [it2])  ; 
pp[iz]  [ix]  [it3]  =  (2  *  pp[2.z]  [ix]  [it2] 

-  FP[i2] [ix] [itl] 

+  v[iz] [ix]  *  (ddx  +  ddz) ) ; 

} 

}; 

for  (ix  =1;  ix  <  nx;  ix++) 

for  (iz  =  1;  iz  <  nz;  iz++) 

p[iz] [ix]  =  pp[iz] [ix] [it3] ; 

) 

sploops 

This  program  is  another  synthetic  example  that  tries  to  exploit  one  of  the  loci  of 
parallelization  that  appears  in  the  previous  program.  The  idea  is  to  suppon  what 
could  be  called  coarse-grain  vectorization,  by  which  a  serial  loops  gets  converted  into 
a  series  of  parallel  loops. 

/*  Coarse-grain  vectorization  */ 
void 

iivask_and_add(int  n,  int  b[n]) 

{ 

int  inask[n]; 

int  i  =  0; 

while  (i  <  n)  { 
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/*  form  the  mask  */ 
if  (i  %  2) 

mask[i]  =  1; 
else 

mask[i]  =  0; 

/*  modify  the  target  */ 
if  (mask[i]  &&  i  >  2) 
b[i]  4=  1; 
else 

b[i]  =  0; 
i  4-  1; 

} 

} 


APPENDIX  C 


Graph  Parsing  Via  The  SSFG  Grammar 


Understanding  the  specifics  of  SSFG  parsing  is  not  needed  to  follow  its  use  in  this 
work,  yet  at  least  some  notion  is  needed  to  understand  the  notation  in  the  catalog. 
The  reader  is  again  referred  to  fFKZ76]  for  the  proofs  and  every  detail  I  omit  here. 

SSFG  parsing  begins  by  classifying  every  node  in  the  given  flow  graph  as  either 
a  decision  (a  node  with  out-degree  2)  or  a  computation  (a  node  with  out-degree  1). 

Then  a  set  of  reuTite  rules  is  applied  eagerly  on  this  graph,  meaning  that,  given 
that  a  rule  matches  locally,  it  can  be  applied  immediately,  regardless  of  other  rules 
possibly  matching  there  or  the  rule  under  consideration  matching  elsewhere.  The 
rules  apply  whenever  a  subgraph  of  the  original  graph  is  isomorphic  to  the  pattern  of 
the  rule.  These  isomorphisms  are  easy  to  handle  by  small,  ad-hoc,  matchers.  One 
can  understand  the  rules  as  the  replacement  of  a  small  subset  of  the  current  graph  by 
an  even  smaller  subgraph. 

Each  rule  converts  the  graph  into  another  with  fewer  nodes  or  fewer  arcs.  Also, 
the  rewriting  rules  form  a  Church-Rosser  system.  All  of  this  guarantees  that  the 
repeated  application  of  these  rewrites  eventually  ends  in  a  unique  resulting  graph. 

If  that  ultimate  graph  contains  exactly  one  node  (which  necessarily  is  then  a 
computation  node),  the  parse  succeeds. 

I  show  here  the  rules  by  means  of  illustrations.  For  each  rule  I  show  the  pattern 
graph  that  triggers  the  reduction,  and  the  reduced  graph. 

Assume  that  the  original  nodes  are  labelled  somehow  (so  that  each  node  is 
distinct).  The  illustrations  that  follow  show  how  the  new  nodes  of  the  reduced  graphs 
gain  labels  derived  from  the  originals,  such  that  the  new  labels  encode  the  reductions 
applied.  Thus  if  the  parsing  finally  succeeds,  the  label  attached  to  the  final  node  is 
actually  a  parse  for  the  original  program. 

Seen  from  the  top  of  Figure  C.  1 ,  the  rules  are  these: 

cc  Computation-computation  elimination.  This  rule  is  essentially  the  T'> 
transformation  in  the  T1-T2  system  [HeU72].  Its  repeated  application 
subsumes  a  linear  segment  of  the  program  (a  generalized  basic  block)  into  one 
node. 

loop  Self-loop  elimination.  This  is  the  corresponding  T1  transformation 

Actually,  the  original  T1  and  T2  transformations  don’t  require  that  there  be  just  one 
exit  node.  The  next  transformations  catch  the  other  cases  that  would  have  been 
reduced  directly  by  either  T1  or  T2.  However,  by  distinguishing  these  subcases,  the 
SSFG  imposes  more  structure  on  the  flow  graph.  Thus,  the  parses  are  richer  in  parts 
than  if  we  just  reduced  segments  and  loops  and  paid  no  attention  to  the  conditional 
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Figure  C.l.  SSFG 
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branching  structure. 

d  Trivial  decision  elimination.  If  at  a  given  step  we  have  modified  the  graph  so 
that  a  decision  (at  A)  doesn’t  matter  any  more  (flow  goes  to  B  anyway),  we 
produce  a  computation  node  that  remembers  the  lost  decision. 

cd  Computation  followed  by  decision.  This  concentrates  the  calculations  leading 
to  a  conditional. 

dc  This  is  the  complement  of  the  above.  Applied  repeatedly  on  a  branch  of  a 
conditional,  it  collects  the  entire  branch  into  a  node. 

dd  Decision-decision  elimination.  Both  conditionals  yield  C  on  occasion,  so  we 
reduce  the  graph  to  remember  the  compound  conditional  tliat  selects  C  versus 
the  rest. 

dioop  This  helps  identifying  cases  of  loops  with  more  than  one  exit.  Eventually  the 
body  will  get  distributed  between  two  decision  nodes,  but  the  reduction  would 
get  stuck  if  this  transformation  were  not  performed. 


